As observed during analysis of MB-51223, almost half of all log messages in the memcached.log are INFO log messages from Magma performing seqno scans - e.g.
2022-02-25T07:52:13.322599-08:00 INFO [(GleamBookUsers0) magma_3/kvstore-907/rev-000000001]0x7f47a9b8ab80 Restarting seqno iterator (184204=>1358962, currSeqno:184205) with seqno 184205 restarts:1
|
Specific line counts:
$ rg -c 'Restarting seqno iterator' memcached.log
|
455346
|
$ wc -l memcached.log
|
990374 memcached.log
|
This limits the temporal extent of our logs and means we only have a short range of timestamps to debug from - in this instance we only have 5 hours worth of history:
$ head -n6 memcached.log
|
==============================================================================
|
Memcached logs
|
sh -c cd "$1"; for file in $(ls -tr memcached.log.*); do cat "$file"; done -- /opt/couchbase/var/lib/couchbase/logs
|
==============================================================================
|
2022-02-25T06:29:18.559127-08:00 INFO ---------- Opening logfile:
|
2022-02-25T06:29:18.558137-08:00 WARNING 7095: (GleamBookUsers0) DCP (Producer) eq_dcpq:secidx:proj-MAINT_STREAM_TOPIC_ae529493879e66deaad5ba003fea5528-1693653966722831168/3 - (vb:789) Stream closing, sent until seqno 305291 remaining items 0, reason: The stream closed early because the conn was disconnected
|
$ tail -n1 memcached.log
|
2022-02-25T11:33:35.840772-08:00 INFO Pushing new global cluster config - revision:{"epoch":1,"revno":5292}
|
Bug
Critical