2022-01-18T11:24:38.955-08:00 [INFO] Current nofiles rlimit: 200000 (max: 200000)
2022-01-18T11:24:38.958-08:00 [INFO] Initialization of cbauth succeeded
2022/01/18 11:24:38 cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/nodeDefs-wanted/e7fbee9d880555f11e932ec1ebb93473, key: nodeDefs-wanted/e7fbee9d880555f11e932ec1ebb93473, deletion: false
2022/01/18 11:24:38 cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/version, key: version, deletion: false
2022/01/18 11:24:38 cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/nodeDefs-known/e7fbee9d880555f11e932ec1ebb93473, key: nodeDefs-known/e7fbee9d880555f11e932ec1ebb93473, deletion: false
2022-01-18T11:24:38.962-08:00 [INFO] New store created with url http://127.0.0.1:8091
2022-01-18T11:24:38.972-08:00 [INFO] keyspace N1QL_SYSTEM_BUCKET not found No bucket named N1QL_SYSTEM_BUCKET
2022-01-18T11:24:38.976-08:00 [INFO] Retrying bucket N1QL_SYSTEM_BUCKET
2022-01-18T11:24:38.978-08:00 [WARN] Changing GC percent from 0 to 200
2022-01-18T11:24:38.982-08:00 [INFO] New settings received: {"timeout":0,"numatrs":1024,"n1ql-feat-ctrl":76,"max-parallelism":1,"query.settings.curl_whitelist":{"all_access":false,"allowed_urls":[],"disallowed_urls":[]},"query.settings.tmp_space_dir":"/opt/couchbase/var/lib/couchbase/tmp","completed-limit":4000,"pipeline-batch":16,"prepared-limit":16384,"pipeline-cap":512,"memory-quota":0,"cleanupwindow":"60s","use-cbo":true,"scan-cap":512,"query.settings.tmp_space_size":5120,"completed-threshold":1000,"loglevel":"info","cleanuplostattempts":true,"cleanupclientattempts":true,"txtimeout":"0ms"}
2022-01-18T11:24:38.982-08:00 [INFO] Settings notifier from metakv
2022-01-18T11:24:38.983-08:00 [INFO] New Value for curl allowedlist map[all_access:false allowed_urls:[] disallowed_urls:[]]
2022-01-18T11:24:38.993-08:00 [INFO] GSI indexer settings have been updated map[query_tmpspace_dir:/opt/couchbase/var/lib/couchbase/tmp query_tmpspace_limit:5120]
2022-01-18T11:24:38.993-08:00 [INFO] n1fty indexer settings have been updated map[query_tmpspace_dir:/opt/couchbase/var/lib/couchbase/tmp query_tmpspace_limit:5120]
2022-01-18T11:24:38.993-08:00 [INFO] Query Configuration changed for txtimeout. New value is 0ms
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for scan-cap. New value is 512
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for cleanupclientattempts. New value is true
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for n1ql-feat-ctrl. New value is 76
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for completed-limit. New value is 4000
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for prepared-limit. New value is 16384
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for pipeline-cap. New value is 512
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for cleanuplostattempts. New value is true
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for max-parallelism. New value is 1
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for memory-quota. New value is 0
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for cleanupwindow. New value is 60s
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for use-cbo. New value is true
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for numatrs. New value is 1024
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for completed-threshold. New value is 1000
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for pipeline-batch. New value is 16
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for timeout. New value is 0
2022-01-18T11:24:38.994-08:00 [INFO] Query Configuration changed for loglevel. New value is info
2022-01-18T11:24:38.993-08:00 [Info] GSIC - Setting config map[query_tmpspace_dir:/opt/couchbase/var/lib/couchbase/tmp query_tmpspace_limit:5120]
2022-01-18T11:24:38.999-08:00 [AUDIT] audit: created new audit service
2022-01-18T11:24:39.009-08:00 [INFO] cbq-engine started version=7.1.0-N1QL datastore=http://127.0.0.1:8091 max-concurrency=4 loglevel=INFO servicers=16 plus-servicers=64 scan-cap=512 pipeline-cap=512 pipeline-batch=16 request-cap=256 request-size-cap=67108864 max-index-api=4 max-parallelism=1 n1ql-feat-ctrl=76 use-cbo=true timeout=0s txtimeout=0s gc-percent=200
2022-01-18T11:24:39.009-08:00 [INFO] Starting audit worker 2
2022-01-18T11:24:39.009-08:00 [INFO] Starting audit worker 3
2022-01-18T11:24:39.009-08:00 [INFO] Starting audit worker 4
2022-01-18T11:24:39.009-08:00 [INFO] Starting audit worker 1
2022-01-18T11:24:39.009-08:00 [INFO] Starting audit settings worker 1.
2022-01-18T11:24:39.011-08:00 [INFO] HttpEndpoint: Listen Address - [::]:8093
2022-01-18T11:24:39.011-08:00 [INFO] HttpEndpoint: Listen Address - 0.0.0.0:8093
2022-01-18T11:24:39.012-08:00 [INFO] Certificates have been refreshed by ns server
2022-01-18T11:24:39.013-08:00 [INFO] HttpEndpoint: ListenTLS Address - 0.0.0.0:18093
2022-01-18T11:24:39.013-08:00 [INFO] HttpEndpoint: ListenTLS Address - [::]:18093
2022-01-18T11:24:39.013-08:00 [INFO] Updating node-to-node encryption level: {EncryptData:false DisableNonSSLPorts:false}
2022-01-18T11:24:39.026-08:00 [INFO] Unable to initialize functions cache monitor Could not access functions change counter because
2022-01-18T11:24:39.973-08:00 [INFO] cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/nodeDefs-known/e7fbee9d880555f11e932ec1ebb93473, key: nodeDefs-known/e7fbee9d880555f11e932ec1ebb93473, deletion: false
2022-01-18T11:24:39.979-08:00 [INFO] cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/nodeDefs-wanted/e7fbee9d880555f11e932ec1ebb93473, key: nodeDefs-wanted/e7fbee9d880555f11e932ec1ebb93473, deletion: false
2022-01-18T11:24:40.010-08:00 [INFO] Starting audit update stream
2022-01-18T11:24:40.153-08:00 [INFO] Initial topology: no active nodes
2022-01-18T11:24:47.736-08:00 [INFO] Topology changed to: e7fbee9d880555f11e932ec1ebb93473[127.0.0.1:8091]
2022-01-18T11:28:04.990-08:00 [INFO] cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/nodeDefs-known/e7fbee9d880555f11e932ec1ebb93473, key: nodeDefs-known/e7fbee9d880555f11e932ec1ebb93473, deletion: false
2022-01-18T11:28:05.011-08:00 [INFO] cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/nodeDefs-wanted/e7fbee9d880555f11e932ec1ebb93473, key: nodeDefs-wanted/e7fbee9d880555f11e932ec1ebb93473, deletion: false
2022-01-18T11:28:05.502-08:00 [INFO] Certificates have been refreshed by ns server
2022-01-18T11:28:05.502-08:00 [INFO] HttpEndpoint: close listener, Address - 0.0.0.0:18093, Error -
2022-01-18T11:28:05.502-08:00 [INFO] HttpEndpoint: close listener, Address - [::]:18093, Error -
2022-01-18T11:28:05.502-08:00 [INFO] HttpEndpoint: ListenTLS Address - 0.0.0.0:18093
2022-01-18T11:28:05.502-08:00 [INFO] HttpEndpoint: ListenTLS Address - [::]:18093
2022-01-18T11:28:32.710-08:00 [INFO] Topology changed to: e7fbee9d880555f11e932ec1ebb93473[172.23.104.90:8091] bcc05a01550931a5547d57933fd62bcc[172.23.104.91:8091]
2022-01-18T11:29:03.311-08:00 [INFO] Created New Bucket default
2022-01-18T11:29:03.312-08:00 [INFO] Trying with http://127.0.0.1:8091/pools/default/bucketsStreaming/default
2022-01-18T11:31:07.375-08:00 [Info] creating GsiClient for 127.0.0.1:8091
2022-01-18T11:31:07.375-08:00 [Info] Recieve security change notification. encryption=false
2022-01-18T11:31:07.375-08:00 [Info] Certificate refreshed successfully with certFile /opt/couchbase/var/lib/couchbase/config/certs/chain.pem, keyFile /opt/couchbase/var/lib/couchbase/config/certs/pkey.pem, caFile /opt/couchbase/var/lib/couchbase/config/certs/ca.pem
2022-01-18T11:31:07.375-08:00 [Info] InitSystemEventLogger: Started SystemEventLogger
2022-01-18T11:31:07.380-08:00 [Info] New settings received:
{"indexer.plasma.backIndex.enablePageBloomFilter":false,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.enable_page_bloom_filter":false,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.log_level":"info","indexer.settings.max_cpu_percent":0,"indexer.settings.memory_quota":268435456,"indexer.settings.num_replica":0,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.storage_mode":"plasma"}
2022-01-18T11:31:07.409-08:00 [Info] security context initialized
2022-01-18T11:31:07.409-08:00 [Info] security port mapping updated : map[11210:11207 8091:18091 8093:18093 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9999:9999]
2022-01-18T11:31:07.418-08:00 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 6
2022-01-18T11:31:07.440-08:00 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 0
2022-01-18T11:31:07.440-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:07.440-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:07.440-08:00 [Info] MetadataProvider.WatchMetadata(): indexer 172.23.104.90:9100
2022-01-18T11:31:07.455-08:00 [Info] WatchMetadata(): successfully reach indexer at 172.23.104.90:9100.
2022-01-18T11:31:07.455-08:00 [Info] MetadataProvider.WatchMetadata(): indexer 172.23.104.91:9100
2022-01-18T11:31:07.474-08:00 [Info] WatchMetadata(): successfully reach indexer at 172.23.104.91:9100.
2022-01-18T11:31:07.476-08:00 [Info] MetadataProvider: Updating indexer version to 6
2022-01-18T11:31:07.476-08:00 [Info] initialized currmeta 2 force true
2022-01-18T11:31:07.478-08:00 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40cbq-engine-cbauth@127.0.0.1:8091, default
2022-01-18T11:31:07.480-08:00 [Info] serviceChangeNotifier: received ServiceChangeNotification
2022-01-18T11:31:07.483-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ...
2022-01-18T11:31:07.483-08:00 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: default
2022-01-18T11:31:07.483-08:00 [Info] [GsiScanClient:"172.23.104.90:9101"] started ...
2022-01-18T11:31:07.483-08:00 [Info] serviceChangeNotifier: received PoolChangeNotification
2022-01-18T11:31:07.484-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] open new connection ...
2022-01-18T11:31:07.488-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:07.488-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:07.490-08:00 [Info] switched currmeta from 2 -> 2 force true
2022-01-18T11:31:07.497-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ...
2022-01-18T11:31:07.497-08:00 [Info] [GsiScanClient:"172.23.104.91:9101"] started ...
2022-01-18T11:31:07.497-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:07.497-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:07.498-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] open new connection ...
2022-01-18T11:31:07.499-08:00 [Info] switched currmeta from 2 -> 2 force true
2022-01-18T11:31:07.502-08:00 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2022-01-18T11:31:07.502-08:00 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2022-01-18T11:31:07.502-08:00 [Info] GSIC[default/default-_default-_default-1642534267371711256] started ...
2022-01-18T11:31:07.502-08:00 [Info] Recieve security change notification. encryption=false
2022-01-18T11:31:07.502-08:00 [Info] Certificate refreshed successfully with certFile /opt/couchbase/var/lib/couchbase/config/certs/chain.pem, keyFile /opt/couchbase/var/lib/couchbase/config/certs/pkey.pem, caFile /opt/couchbase/var/lib/couchbase/config/certs/ca.pem
2022-01-18T11:31:07.502-08:00 [INFO] n1fty: NewFTSIndexer, server: http://127.0.0.1:8091, namespace: default, keyspace: default
2022-01-18T11:31:07.510-08:00 [Info] CreateIndex default _default _default #primary ...
2022-01-18T11:31:07.511-08:00 [Info] send prepare create request to watcher 172.23.104.91:9100
2022-01-18T11:31:07.511-08:00 [Info] send prepare create request to watcher 172.23.104.90:9100
2022-01-18T11:31:07.513-08:00 [Info] Indexer 172.23.104.91:9100 accept prepare request. Index (default, _default, _default, #primary)
2022-01-18T11:31:07.513-08:00 [Info] Indexer 172.23.104.90:9100 accept prepare request. Index (default, _default, _default, #primary)
2022-01-18T11:31:07.609-08:00 [Info] Planner::processCreateToken: There is no create token to process for node: 172.23.104.90:8091
2022-01-18T11:31:07.609-08:00 [Info] Planner::processCreateToken: There is no create token to process for node: 172.23.104.91:8091
2022-01-18T11:31:07.611-08:00 [Info] Planner::processDeleteToken: There is no delete token to process for node: 172.23.104.91:8091
2022-01-18T11:31:07.612-08:00 [Info] Planner::processDeleteToken: There is no delete token to process for node: 172.23.104.90:8091
2022-01-18T11:31:07.614-08:00 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 172.23.104.90:8091
2022-01-18T11:31:07.614-08:00 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 172.23.104.91:8091
2022-01-18T11:31:07.623-08:00 [Info] Using greedy index placement for index 9569406329691548240
2022-01-18T11:31:07.623-08:00 [Info] ************ Indexer Layout *************
2022-01-18T11:31:07.623-08:00 [Info] Using GreedyPlanner
2022-01-18T11:31:07.623-08:00 [Info] Indexer Memory Mean 0 (0)
2022-01-18T11:31:07.623-08:00 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2022-01-18T11:31:07.623-08:00 [Info] Indexer Memory Utilization 0.0000
2022-01-18T11:31:07.623-08:00 [Info] Indexer CPU Mean 0.0000
2022-01-18T11:31:07.623-08:00 [Info] Indexer CPU Deviation 0.00 (0.00%)
2022-01-18T11:31:07.623-08:00 [Info] Indexer CPU Utilization 0.0000
2022-01-18T11:31:07.623-08:00 [Info] Indexer IO Mean 0.0000
2022-01-18T11:31:07.623-08:00 [Info] Indexer IO Deviation 0.00 (0.00%)
2022-01-18T11:31:07.623-08:00 [Info] Indexer Drain Rate Mean 0.0000
2022-01-18T11:31:07.623-08:00 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2022-01-18T11:31:07.623-08:00 [Info] Indexer Scan Rate Mean 0.0000
2022-01-18T11:31:07.623-08:00 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2022-01-18T11:31:07.623-08:00 [Info] Indexer Data Size Mean 0 (0)
2022-01-18T11:31:07.623-08:00 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2022-01-18T11:31:07.623-08:00 [Info] Total Index Data (from non-deleted node) 0
2022-01-18T11:31:07.623-08:00 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2022-01-18T11:31:07.623-08:00 [Info] No. Index (from non-deleted node) 0
2022-01-18T11:31:07.623-08:00 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2022-01-18T11:31:07.623-08:00 [Info] ----------------------------------------
2022-01-18T11:31:07.623-08:00 [Info] Number of indexes: 1
2022-01-18T11:31:07.623-08:00 [Info] Number of indexers: 2
2022-01-18T11:31:07.623-08:00 [Info] Avg Index Size: 0 (0)
2022-01-18T11:31:07.623-08:00 [Info] Max Index Size: 0 (0)
2022-01-18T11:31:07.623-08:00 [Info] Max Indexer Overhead: 0 (0)
2022-01-18T11:31:07.623-08:00 [Info] Avg Index Cpu: 0.0000
2022-01-18T11:31:07.623-08:00 [Info] Max Index Cpu: 0.0000
2022-01-18T11:31:07.623-08:00 [Info] Num Estimation Run: 1
2022-01-18T11:31:07.623-08:00 [Info] ----------------------------------------
2022-01-18T11:31:07.623-08:00 [Info] Memory Quota 268435456 (256M)
2022-01-18T11:31:07.623-08:00 [Info] CPU Quota 4
2022-01-18T11:31:07.623-08:00 [Info] Max Cpu Utilization -1
2022-01-18T11:31:07.623-08:00 [Info] Max Memory Utilization -1
2022-01-18T11:31:07.623-08:00 [Info] ----------------------------------------
2022-01-18T11:31:07.623-08:00 [Info] Total iteration: 0
2022-01-18T11:31:07.623-08:00 [Info] RandomSwap time: 0ns
2022-01-18T11:31:07.623-08:00 [Info] RandomSwap call: 0
2022-01-18T11:31:07.623-08:00 [Info] RandomSwap iteration: 0
2022-01-18T11:31:07.623-08:00 [Info] RandomMove time: 0ns
2022-01-18T11:31:07.623-08:00 [Info] RandomMove call: 0 (empty index 0)
2022-01-18T11:31:07.623-08:00 [Info] ExhaustMove time: 0ns
2022-01-18T11:31:07.623-08:00 [Info] ExhaustMove call: 0
2022-01-18T11:31:07.623-08:00 [Info] ExhaustSwap time: 0ns
2022-01-18T11:31:07.623-08:00 [Info] ExhaustSwap call: 0
2022-01-18T11:31:07.623-08:00 [Info] ----------------------------------------
2022-01-18T11:31:07.623-08:00 [Info]
2022-01-18T11:31:07.623-08:00 [Info] Indexer serverGroup:Group 1, nodeId:172.23.104.91:8091, nodeUUID:bcc05a01550931a5547d57933fd62bcc, useLiveData:true
2022-01-18T11:31:07.623-08:00 [Info] Indexer total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2022-01-18T11:31:07.623-08:00 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0
2022-01-18T11:31:07.623-08:00 [Info] ------------------------------------------------------------------------------------------------------------------
2022-01-18T11:31:07.623-08:00 [Info] Index name:#primary 0, bucket:default, scope:_default, collection:_default, defnId:9569406329691548240, instId:13800870380479402930, Partition: 0, new/moved:true
2022-01-18T11:31:07.623-08:00 [Info] Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2022-01-18T11:31:07.623-08:00 [Info] Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2022-01-18T11:31:07.623-08:00 [Info]
2022-01-18T11:31:07.623-08:00 [Info] Indexer serverGroup:Group 1, nodeId:172.23.104.90:8091, nodeUUID:e7fbee9d880555f11e932ec1ebb93473, useLiveData:true
2022-01-18T11:31:07.623-08:00 [Info] Indexer total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:0
2022-01-18T11:31:07.624-08:00 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0
2022-01-18T11:31:07.624-08:00 [Info] ****************************************
2022-01-18T11:31:07.624-08:00 [Info] send commit create request to watcher 172.23.104.91:9100 defnID 9569406329691548240
2022-01-18T11:31:07.624-08:00 [Info] send commit create request to watcher 172.23.104.90:9100 defnID 9569406329691548240
2022-01-18T11:31:07.673-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:07.673-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:07.675-08:00 [Info] switched currmeta from 2 -> 4 force true
2022-01-18T11:31:07.691-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:07.692-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:07.693-08:00 [Info] switched currmeta from 4 -> 6 force true
2022-01-18T11:31:07.696-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:07.696-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:07.697-08:00 [Info] switched currmeta from 6 -> 6 force true
2022-01-18T11:31:07.923-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:07.923-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:07.927-08:00 [Info] switched currmeta from 6 -> 7 force true
2022-01-18T11:31:10.532-08:00 [Info] CreateIndex 9569406329691548240 default _default _default/#primary using:GSI exprType:N1QL whereExpr:() secExprs:([]) desc:[] isPrimary:true scheme:SINGLE partitionKeys:([]) with: - elapsed(3.022013854s) err()
2022-01-18T11:31:10.532-08:00 [Info] metadata provider version changed 7 -> 8
2022-01-18T11:31:10.532-08:00 [Info] switched currmeta from 7 -> 8 force false
2022-01-18T11:31:10.533-08:00 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2022-01-18T11:31:10.538-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:10.538-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:10.541-08:00 [Info] switched currmeta from 8 -> 8 force true
2022-01-18T11:31:12.610-08:00 [Info] serviceChangeNotifier: received PoolChangeNotification
2022-01-18T11:31:12.631-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:12.631-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:12.636-08:00 [Info] switched currmeta from 8 -> 8 force true
2022-01-18T11:31:16.201-08:00 [Info] Rollback time has changed for index inst 3492718749053925295. New rollback time 1642534267681978921
2022-01-18T11:31:46.214-08:00 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2022-01-18T11:31:46.217-08:00 [Info] CreateIndex default _default _default adv_job_title ...
2022-01-18T11:31:46.217-08:00 [Info] send prepare create request to watcher 172.23.104.90:9100
2022-01-18T11:31:46.217-08:00 [Info] send prepare create request to watcher 172.23.104.91:9100
2022-01-18T11:31:46.218-08:00 [Info] Indexer 172.23.104.90:9100 accept prepare request. Index (default, _default, _default, adv_job_title)
2022-01-18T11:31:46.218-08:00 [Info] Indexer 172.23.104.91:9100 accept prepare request. Index (default, _default, _default, adv_job_title)
2022-01-18T11:31:46.259-08:00 [Info] Planner::processCreateToken: There is no create token to process for node: 172.23.104.91:8091
2022-01-18T11:31:46.259-08:00 [Info] Planner::processCreateToken: There is no create token to process for node: 172.23.104.90:8091
2022-01-18T11:31:46.263-08:00 [Info] Planner::processDeleteToken: There is no delete token to process for node: 172.23.104.90:8091
2022-01-18T11:31:46.263-08:00 [Info] Planner::processDeleteToken: There is no delete token to process for node: 172.23.104.91:8091
2022-01-18T11:31:46.266-08:00 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 172.23.104.90:8091
2022-01-18T11:31:46.266-08:00 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 172.23.104.91:8091
2022-01-18T11:31:46.278-08:00 [Info] Using greedy index placement for index 778662462714092586
2022-01-18T11:31:46.278-08:00 [Info] ************ Indexer Layout *************
2022-01-18T11:31:46.278-08:00 [Info] Using GreedyPlanner
2022-01-18T11:31:46.278-08:00 [Info] Indexer Memory Mean 63892 (62.3945K)
2022-01-18T11:31:46.278-08:00 [Info] Indexer Memory Deviation 63892 (62.3945K) (100.00%)
2022-01-18T11:31:46.278-08:00 [Info] Indexer Memory Utilization 0.0002
2022-01-18T11:31:46.278-08:00 [Info] Indexer CPU Mean 0.0000
2022-01-18T11:31:46.278-08:00 [Info] Indexer CPU Deviation 0.00 (0.00%)
2022-01-18T11:31:46.278-08:00 [Info] Indexer CPU Utilization 0.0000
2022-01-18T11:31:46.278-08:00 [Info] Indexer IO Mean 0.0000
2022-01-18T11:31:46.278-08:00 [Info] Indexer IO Deviation 0.00 (0.00%)
2022-01-18T11:31:46.278-08:00 [Info] Indexer Drain Rate Mean 0.0000
2022-01-18T11:31:46.278-08:00 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2022-01-18T11:31:46.278-08:00 [Info] Indexer Scan Rate Mean 0.0000
2022-01-18T11:31:46.278-08:00 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2022-01-18T11:31:46.278-08:00 [Info] Indexer Data Size Mean 24072 (23.5078K)
2022-01-18T11:31:46.278-08:00 [Info] Indexer Data Size Deviation 24072 (23.5078K) (100.00%)
2022-01-18T11:31:46.278-08:00 [Info] Total Index Data (from non-deleted node) 0
2022-01-18T11:31:46.278-08:00 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2022-01-18T11:31:46.278-08:00 [Info] No. Index (from non-deleted node) 0
2022-01-18T11:31:46.278-08:00 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2022-01-18T11:31:46.278-08:00 [Info] ----------------------------------------
2022-01-18T11:31:46.278-08:00 [Info] Number of indexes: 2
2022-01-18T11:31:46.278-08:00 [Info] Number of indexers: 2
2022-01-18T11:31:46.278-08:00 [Info] Avg Index Size: 63892 (62.3945K)
2022-01-18T11:31:46.278-08:00 [Info] Max Index Size: 127784 (124.789K)
2022-01-18T11:31:46.278-08:00 [Info] Max Indexer Overhead: 144338704 (137.652M)
2022-01-18T11:31:46.278-08:00 [Info] Avg Index Cpu: 0.0000
2022-01-18T11:31:46.278-08:00 [Info] Max Index Cpu: 0.0000
2022-01-18T11:31:46.278-08:00 [Info] Num Estimation Run: 1
2022-01-18T11:31:46.278-08:00 [Info] ----------------------------------------
2022-01-18T11:31:46.278-08:00 [Info] Memory Quota 268435456 (256M)
2022-01-18T11:31:46.278-08:00 [Info] CPU Quota 4
2022-01-18T11:31:46.278-08:00 [Info] Max Cpu Utilization -1
2022-01-18T11:31:46.278-08:00 [Info] Max Memory Utilization -1
2022-01-18T11:31:46.278-08:00 [Info] ----------------------------------------
2022-01-18T11:31:46.278-08:00 [Info] Total iteration: 0
2022-01-18T11:31:46.278-08:00 [Info] RandomSwap time: 0ns
2022-01-18T11:31:46.278-08:00 [Info] RandomSwap call: 0
2022-01-18T11:31:46.278-08:00 [Info] RandomSwap iteration: 0
2022-01-18T11:31:46.278-08:00 [Info] RandomMove time: 0ns
2022-01-18T11:31:46.278-08:00 [Info] RandomMove call: 0 (empty index 0)
2022-01-18T11:31:46.278-08:00 [Info] ExhaustMove time: 0ns
2022-01-18T11:31:46.278-08:00 [Info] ExhaustMove call: 0
2022-01-18T11:31:46.278-08:00 [Info] ExhaustSwap time: 0ns
2022-01-18T11:31:46.278-08:00 [Info] ExhaustSwap call: 0
2022-01-18T11:31:46.278-08:00 [Info] ----------------------------------------
2022-01-18T11:31:46.278-08:00 [Info]
2022-01-18T11:31:46.278-08:00 [Info] Indexer serverGroup:Group 1, nodeId:172.23.104.90:8091, nodeUUID:e7fbee9d880555f11e932ec1ebb93473, useLiveData:true
2022-01-18T11:31:46.278-08:00 [Info] Indexer total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2022-01-18T11:31:46.278-08:00 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0
2022-01-18T11:31:46.278-08:00 [Info] ------------------------------------------------------------------------------------------------------------------
2022-01-18T11:31:46.278-08:00 [Info] Index name:adv_job_title 0, bucket:default, scope:_default, collection:_default, defnId:778662462714092586, instId:2710802142558000415, Partition: 0, new/moved:true
2022-01-18T11:31:46.278-08:00 [Info] Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2022-01-18T11:31:46.278-08:00 [Info] Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2022-01-18T11:31:46.278-08:00 [Info]
2022-01-18T11:31:46.278-08:00 [Info] Indexer serverGroup:Group 1, nodeId:172.23.104.91:8091, nodeUUID:bcc05a01550931a5547d57933fd62bcc, useLiveData:true
2022-01-18T11:31:46.278-08:00 [Info] Indexer total memory:144466488 (137.774M), mem:127784 (124.789K), overhead:144338704 (137.652M), min:28893297 (27.5548M), data:48145 (47.0166K) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2022-01-18T11:31:46.278-08:00 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.8
2022-01-18T11:31:46.278-08:00 [Info] ------------------------------------------------------------------------------------------------------------------
2022-01-18T11:31:46.278-08:00 [Info] Index name:#primary 0, bucket:default, scope:_default, collection:_default, defnId:9569406329691548240, instId:3492718749053925295, Partition: 0, new/moved:false
2022-01-18T11:31:46.278-08:00 [Info] Index total memory:144466488 (137.774M), mem:127784 (124.789K), overhead:144338704 (137.652M), min:28893297 (27.5548M), data:48145 (47.0166K) cpu:0.0000 io:0 (0) scan:0 drain:0
2022-01-18T11:31:46.278-08:00 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2022-01-18T11:31:46.278-08:00 [Info] ****************************************
2022-01-18T11:31:46.278-08:00 [Info] send commit create request to watcher 172.23.104.90:9100 defnID 778662462714092586
2022-01-18T11:31:46.279-08:00 [Info] send commit create request to watcher 172.23.104.91:9100 defnID 778662462714092586
2022-01-18T11:31:46.298-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:46.298-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:46.300-08:00 [Info] switched currmeta from 8 -> 10 force true
2022-01-18T11:31:46.332-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:46.332-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:46.333-08:00 [Info] switched currmeta from 10 -> 12 force true
2022-01-18T11:31:46.338-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:46.338-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:46.341-08:00 [Info] switched currmeta from 12 -> 12 force true
2022-01-18T11:31:46.539-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:46.539-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:46.541-08:00 [Info] switched currmeta from 12 -> 13 force true
2022-01-18T11:31:47.660-08:00 [Info] serviceChangeNotifier: received PoolChangeNotification
2022-01-18T11:31:47.681-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:47.681-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:47.682-08:00 [Info] switched currmeta from 13 -> 13 force true
2022-01-18T11:31:49.032-08:00 [Info] CreateIndex 778662462714092586 default _default _default/adv_job_title using:GSI exprType:N1QL whereExpr:() secExprs:([`job_title`]) desc:[false] isPrimary:false scheme:SINGLE partitionKeys:([]) with: - elapsed(2.81488357s) err()
2022-01-18T11:31:49.032-08:00 [Info] metadata provider version changed 13 -> 14
2022-01-18T11:31:49.032-08:00 [Info] switched currmeta from 13 -> 14 force false
2022-01-18T11:31:49.032-08:00 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2022-01-18T11:31:49.048-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:49.048-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:49.050-08:00 [Info] switched currmeta from 14 -> 14 force true
2022-01-18T11:31:52.669-08:00 [Info] serviceChangeNotifier: received PoolChangeNotification
2022-01-18T11:31:52.676-08:00 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2022-01-18T11:31:52.676-08:00 [Info] Refreshed Indexer List: [172.23.104.90:9100 172.23.104.91:9100]
2022-01-18T11:31:52.678-08:00 [Info] switched currmeta from 14 -> 14 force true
2022-01-18T11:31:54.473-08:00 [Info] Rollback time has changed for index inst 5141564385820382215. New rollback time 1642534306321073167
2022-01-18T11:32:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:32:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:32:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:32:07.504-08:00 [Info] average scan response {0 ms}
2022-01-18T11:32:07.522-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:32:07.539-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:33:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:33:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:33:07.506-08:00 [Info] num concurrent scans {0}
2022-01-18T11:33:07.507-08:00 [Info] average scan response {0 ms}
2022-01-18T11:33:07.559-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:33:07.581-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:34:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:34:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:34:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:34:07.504-08:00 [Info] average scan response {0 ms}
2022-01-18T11:34:07.595-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:34:07.620-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:35:07.476-08:00 [Info] connected with 2 indexers
2022-01-18T11:35:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:35:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:35:07.504-08:00 [Info] average scan response {0 ms}
2022-01-18T11:35:07.633-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:35:07.659-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:36:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:36:07.478-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:36:07.503-08:00 [Info] num concurrent scans {0}
2022-01-18T11:36:07.503-08:00 [Info] average scan response {0 ms}
2022-01-18T11:36:07.673-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:36:07.700-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:37:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:37:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:37:07.503-08:00 [Info] num concurrent scans {0}
2022-01-18T11:37:07.503-08:00 [Info] average scan response {0 ms}
2022-01-18T11:37:07.716-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:37:07.735-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:38:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:38:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:38:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:38:07.504-08:00 [Info] average scan response {0 ms}
2022-01-18T11:38:07.753-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:38:07.773-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:39:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:39:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:39:07.503-08:00 [Info] num concurrent scans {0}
2022-01-18T11:39:07.503-08:00 [Info] average scan response {0 ms}
2022-01-18T11:39:07.792-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:39:07.807-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:39:37.457-08:00 [INFO] Transaction Initialization: ExpirationTime: 15s, CleanupWindow: 1m0s, CleanupClientAttempts: true, CleanupLostAttempts: true
2022-01-18T11:39:37.463-08:00 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
2022-01-18T11:39:37.474-08:00 [Info] GSIC[default/default-_default-_default-1642534267371711256] 9d767a4f-ca6f-41eb-94f5-005f9c2f77a8 new temp file ... /opt/couchbase/var/lib/couchbase/tmp/scan-results10737915070512
2022-01-18T11:39:37.474-08:00 [INFO] (TXGOCBCORE) SDK Version: gocbcore/v10.0.6
2022-01-18T11:39:37.474-08:00 [INFO] (TXGOCBCORE) Creating new agent: {BucketName:default UserAgent:default SeedConfig:{HTTPAddrs:[127.0.0.1:8091] MemdAddrs:[]} SecurityConfig:{UseTLS:false TLSRootCAProvider:0x2050fa0 NoTLSSeedNode:true Auth:0xc003f921a8 AuthMechanisms:[]} CompressionConfig:{Enabled:false DisableDecompression:false MinSize:0 MinRatio:0} ConfigPollerConfig:{HTTPRedialPeriod:0s HTTPRetryDelay:0s HTTPMaxWait:0s CccpMaxWait:0s CccpPollPeriod:0s} IoConfig:{NetworkType:default UseMutationTokens:false UseDurations:false UseOutOfOrderResponses:false DisableXErrorHello:false DisableJSONHello:false DisableSyncReplicationHello:false EnablePITRHello:false UseCollections:true} KVConfig:{ConnectTimeout:10s ServerWaitBackoff:0s PoolSize:8 MaxQueueSize:32768} HTTPConfig:{MaxIdleConns:0 MaxIdleConnsPerHost:0 IdleConnectionTimeout:0s} DefaultRetryStrategy:0xc003f921a0 CircuitBreakerConfig:{Enabled:false VolumeThreshold:0 ErrorThresholdPercentage:0 SleepWindow:0s RollingWindow:0s CompletionCallback: CanaryTimeout:0s} OrphanReporterConfig:{Enabled:false ReportInterval:0s SampleSize:0} TracerConfig:{Tracer: NoRootTraceSpans:false} MeterConfig:{Meter:}}
2022-01-18T11:39:37.566-08:00 [SEVERE] panic: interface conversion: interface {} is nil, not *couchbase.TransactionMutations
2022-01-18T11:39:37.566-08:00 [SEVERE] request text: UPDATE default SET job_title = "ENGINEER" WHERE join_yr = 2011 AND join_mo = 10 AND lower(job_title) = "engineer" RETURNING name, job_title;
2022-01-18T11:39:37.566-08:00 [SEVERE] query context:
2022-01-18T11:39:37.566-08:00 [SEVERE] stack: goroutine 916 [running]:
github.com/couchbase/query/execution.(*Context).Recover(0xc004286a00, 0xc00088b8c0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/context.go:1040 +0xc6
panic(0x23096a0, 0xc0005fc030)
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.16.6/go/src/runtime/panic.go:965 +0x1b9
github.com/couchbase/query/execution.(*base).runConsumer.func1.2(0x1, 0xc00088b8c0, 0xc004286a00)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:888 +0xdc
panic(0x23096a0, 0xc0005fc030)
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.16.6/go/src/runtime/panic.go:965 +0x1b9
github.com/couchbase/query/datastore/couchbase.(*keyspace).txPerformOp(0xc000372000, 0x3, 0xc000b97920, 0x21, 0xc0008621a8, 0x8, 0xc0008621a0, 0x8, 0x0, 0x0, ...)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/datastore/couchbase/txcouchbase.go:513 +0x1832
github.com/couchbase/query/datastore/couchbase.(*keyspace).performOp(0xc000372000, 0x3, 0xc000b97920, 0x21, 0xc0008621a8, 0x8, 0xc0008621a0, 0x8, 0xc000d72000, 0x1, ...)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/datastore/couchbase/couchbase.go:1907 +0x172
github.com/couchbase/query/datastore/couchbase.(*collection).Update(0xc000891d40, 0xc000d72000, 0x1, 0x200, 0x288dc00, 0xc004286a00, 0x0, 0x1, 0x1, 0x0, ...)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/datastore/couchbase/collections.go:328 +0x196
github.com/couchbase/query/execution.(*SendUpdate).flushBatch(0xc00088b8c0, 0xc004286a00, 0x0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/update_send.go:188 +0x682
github.com/couchbase/query/execution.(*base).enbatchSize(0xc00088b8c0, 0x28db170, 0xc0040a7d40, 0x28768a0, 0xc00088b8c0, 0x0, 0xc004286a00, 0xc0040a7d00, 0x28db170)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:1197 +0xda
github.com/couchbase/query/execution.(*base).enbatch(...)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:1210
github.com/couchbase/query/execution.(*SendUpdate).processItem(0xc00088b8c0, 0x28db170, 0xc0040a7d40, 0xc004286a00, 0xc0040a7d40)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/update_send.go:72 +0x9f
github.com/couchbase/query/execution.(*base).runConsumer.func1()
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:923 +0x1c2
github.com/couchbase/query/util.(*Once).Do(0xc00088b9f8, 0xc000625f00)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/util/sync.go:54 +0x4b
github.com/couchbase/query/execution.(*base).runConsumer(0xc00088b8c0, 0x2873038, 0xc00088b8c0, 0xc004286a00, 0x0, 0x0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:857 +0xa5
github.com/couchbase/query/execution.(*SendUpdate).RunOnce(0xc00088b8c0, 0xc004286a00, 0x0, 0x0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/update_send.go:68 +0x5c
github.com/couchbase/query/execution.execOp(0x28a20c8, 0xc00088b8c0, 0xc004286a00, 0x0, 0x0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:519 +0x57
created by github.com/couchbase/query/execution.(*base).fork
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:530 +0x105
goroutine 916 [running]:
github.com/couchbase/query/execution.(*Context).Recover(0xc004286a00, 0xc00088b8c0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/context.go:1040 +0xc6
panic(0x23096a0, 0xc0005fc030)
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.16.6/go/src/runtime/panic.go:965 +0x1b9
github.com/couchbase/query/execution.(*base).runConsumer.func1.2(0x1, 0xc00088b8c0, 0xc004286a00)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:888 +0xdc
panic(0x23096a0, 0xc0005fc030)
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.16.6/go/src/runtime/panic.go:965 +0x1b9
github.com/couchbase/query/datastore/couchbase.(*keyspace).txPerformOp(0xc000372000, 0x3, 0xc000b97920, 0x21, 0xc0008621a8, 0x8, 0xc0008621a0, 0x8, 0x0, 0x0, ...)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/datastore/couchbase/txcouchbase.go:513 +0x1832
github.com/couchbase/query/datastore/couchbase.(*keyspace).performOp(0xc000372000, 0x3, 0xc000b97920, 0x21, 0xc0008621a8, 0x8, 0xc0008621a0, 0x8, 0xc000d72000, 0x1, ...)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/datastore/couchbase/couchbase.go:1907 +0x172
github.com/couchbase/query/datastore/couchbase.(*collection).Update(0xc000891d40, 0xc000d72000, 0x1, 0x200, 0x288dc00, 0xc004286a00, 0x0, 0x1, 0x1, 0x0, ...)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/datastore/couchbase/collections.go:328 +0x196
github.com/couchbase/query/execution.(*SendUpdate).flushBatch(0xc00088b8c0, 0xc004286a00, 0x0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/update_send.go:188 +0x682
github.com/couchbase/query/execution.(*base).enbatchSize(0xc00088b8c0, 0x28db170, 0xc0040a7d40, 0x28768a0, 0xc00088b8c0, 0x0, 0xc004286a00, 0xc0040a7d00, 0x28db170)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:1197 +0xda
github.com/couchbase/query/execution.(*base).enbatch(...)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:1210
github.com/couchbase/query/execution.(*SendUpdate).processItem(0xc00088b8c0, 0x28db170, 0xc0040a7d40, 0xc004286a00, 0xc0040a7d40)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/update_send.go:72 +0x9f
github.com/couchbase/query/execution.(*base).runConsumer.func1()
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:923 +0x1c2
github.com/couchbase/query/util.(*Once).Do(0xc00088b9f8, 0xc000625f00)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/util/sync.go:54 +0x4b
github.com/couchbase/query/execution.(*base).runConsumer(0xc00088b8c0, 0x2873038, 0xc00088b8c0, 0xc004286a00, 0x0, 0x0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:857 +0xa5
github.com/couchbase/query/execution.(*SendUpdate).RunOnce(0xc00088b8c0, 0xc004286a00, 0x0, 0x0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/update_send.go:68 +0x5c
github.com/couchbase/query/execution.execOp(0x28a20c8, 0xc00088b8c0, 0xc004286a00, 0x0, 0x0)
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:519 +0x57
created by github.com/couchbase/query/execution.(*base).fork
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/execution/base.go:530 +0x105
2022-01-18T11:39:37.566-08:00 [Info] GSIC[default/default-_default-_default-1642534267371711256] request(9d767a4f-ca6f-41eb-94f5-005f9c2f77a8) removing temp file /opt/couchbase/var/lib/couchbase/tmp/scan-results10737915070512 ...
2022-01-18T11:39:47.479-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:39:57.480-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:40:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:40:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:40:07.481-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:40:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:40:07.504-08:00 [Info] average scan response {49 ms}
2022-01-18T11:40:07.504-08:00 [Info] GSIC[default/default-_default-_default-1642534267371711256] logstats "default" {"gsi_scan_count":1,"gsi_scan_duration":98909197,"gsi_throttle_duration":719982,"gsi_prime_duration":4859456,"gsi_blocked_duration":64193050,"gsi_total_temp_files":1}
2022-01-18T11:40:07.831-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:40:07.846-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:40:17.481-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:40:27.482-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:40:37.482-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:40:47.483-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:40:57.484-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:41:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:41:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:41:07.484-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:41:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:41:07.504-08:00 [Info] average scan response {49 ms}
2022-01-18T11:41:07.868-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:41:07.884-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:41:17.484-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:41:27.485-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:41:37.485-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:41:47.486-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:41:57.486-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:42:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:42:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:42:07.487-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:42:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:42:07.504-08:00 [Info] average scan response {49 ms}
2022-01-18T11:42:07.905-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:42:07.917-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:42:17.487-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:42:27.488-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:42:37.488-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:42:47.489-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:42:57.489-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:43:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:43:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:43:07.490-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:43:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:43:07.504-08:00 [Info] average scan response {49 ms}
2022-01-18T11:43:07.945-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:43:07.957-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:43:17.491-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:43:27.491-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:43:37.492-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:43:47.492-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:43:57.494-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:44:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:44:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:44:07.494-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:44:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:44:07.504-08:00 [Info] average scan response {49 ms}
2022-01-18T11:44:07.982-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:44:07.998-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:44:17.495-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:44:27.496-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:44:37.496-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:44:47.497-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:44:57.498-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:45:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:45:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:45:07.499-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:45:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:45:07.504-08:00 [Info] average scan response {49 ms}
2022-01-18T11:45:08.018-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:45:08.035-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:45:17.499-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:45:27.500-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:45:37.501-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:45:47.502-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:45:57.503-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:46:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:46:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:46:07.503-08:00 [Info] num concurrent scans {0}
2022-01-18T11:46:07.503-08:00 [Info] average scan response {49 ms}
2022-01-18T11:46:07.504-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:46:08.059-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:46:08.075-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:46:17.505-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:46:27.506-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:46:37.507-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:46:47.508-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:46:57.509-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:47:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:47:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:47:07.503-08:00 [Info] num concurrent scans {0}
2022-01-18T11:47:07.503-08:00 [Info] average scan response {49 ms}
2022-01-18T11:47:07.510-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:47:08.097-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:47:08.115-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:47:17.511-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:47:27.512-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:47:37.513-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:47:47.514-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:47:57.514-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:48:07.477-08:00 [Info] connected with 2 indexers
2022-01-18T11:48:07.477-08:00 [Info] client stats current counts: current: 2, not current: 0
2022-01-18T11:48:07.504-08:00 [Info] num concurrent scans {0}
2022-01-18T11:48:07.504-08:00 [Info] average scan response {49 ms}
2022-01-18T11:48:07.516-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:48:08.133-08:00 [Info] [Queryport-connpool:172.23.104.90:9101] active conns 0, free conns 1
2022-01-18T11:48:08.157-08:00 [Info] [Queryport-connpool:172.23.104.91:9101] active conns 0, free conns 1
2022-01-18T11:48:17.516-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:48:27.517-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:48:37.518-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:48:47.519-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
2022-01-18T11:48:57.520-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid