Details
-
Bug
-
Resolution: Unresolved
-
Major
-
7.1.1
-
None
-
Untriaged
-
1
-
Unknown
Description
Build 7.1.1-3111
Test : -test tests/2i/neo/test_neo_idx_clusterops_recovery.yml
Scale: 1
Day: 1st and 2nd.
On 172.23.107.4 , we see a flooding of this message
/opt/couchbase/var/lib/couchbase/logs/indexer.log.3.gz:2022-06-08T07:44:23.604-07:00 [Info] Timekeeper::ensureMonotonicTs Align seqno smaller than lastFlushTs. KeyspaceId bucket4 StreamId MAINT_STREAM vbucket 348. CurrentTS: Snapshot 6462-6462 Seqno 6462 Vbuuid 23570400411830. LastFlushTS: Snapshot 6462-6468 Seqno 6468 Vbuuid 23570400411830. HWT [6470-6470, 6470]. LastSnap [6469-6469, 6469]. |
/opt/couchbase/var/lib/couchbase/logs/indexer.log.3.gz:2022-06-08T07:44:23.606-07:00 [Info] Timekeeper::ensureMonotonicTs Align seqno smaller than lastFlushTs. KeyspaceId bucket4 StreamId MAINT_STREAM vbucket 343. CurrentTS: Snapshot 6462-6462 Seqno 6462 Vbuuid 249294778014705. LastFlushTS: Snapshot 6462-6468 Seqno 6468 Vbuuid 249294778014705. HWT [6470-6470, 6470]. LastSnap [6469-6469, 6469]. |
/opt/couchbase/var/lib/couchbase/logs/indexer.log.3.gz:2022-06-08T07:44:23.606-07:00 [Info] Timekeeper::ensureMonotonicTs Align seqno smaller than lastFlushTs. KeyspaceId bucket4 StreamId MAINT_STREAM vbucket 348. CurrentTS: Snapshot 6462-6462 Seqno 6462 Vbuuid 23570400411830. LastFlushTS: Snapshot 6462-6468 Seqno 6468 Vbuuid 23570400411830. HWT [6470-6470, 6470]. LastSnap [6469-6469, 6469]. |
Spoke to Varun Velamuri and he mentioned that if the DCP stream of a KV replica node is behind the DCP stream of an indexer node, it's possible to see this. But at the time of this log, no KV operations are being run . The Jenkins console can be found here
http://172.23.109.231/job/component_systest_launcher_3/447/consoleFull
A similar issue has been raised in the past https://issues.couchbase.com/browse/MB-46318.
During the run, it looks like the tests were running create indexes in a loop.
cbcollect logs:
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.107.3.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.107.4.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.107.5.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.97.215.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.97.216.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.97.217.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.97.227.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.97.232.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.97.235.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.97.236.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1654702305/collectinfo-2022-06-08T153146-ns_1%40172.23.97.237.zip