Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.1.0
-
Untriaged
-
-
1
-
Yes
-
KV 2021-Nov
Description
Build : 7.1.0-1663
Test : -test tests/2i/neo/test_neo_idx_clusterops_recovery.yml -scope tests/2i/neo/scope_neo_plasma_idx_dgm.yml (OSO enabled)
Scale : 2
Iteration : 1st
On the following nodes, 172.23.97.215, 172.23.97.232, 172.23.97.235, 172.23.97.237, between 2021-11-09T00:22:10 and 2021-11-09T00:22:48, in the projector logs, msgs related to seq order violation were seen.
172.23.97.215 : projector
|
/opt/couchbase/var/lib/couchbase/logs/projector.log:2021-11-09T00:22:11.040-08:00 [Fatal] DCPT[secidx:proj-bucket3-MAINT_STREAM_TOPIC_799c07c811d6f05cef27c3dba8db6676-3190590407978436790/0] seq order violation for vb = 72, seq = 26655, opcode = DCP_SEQNO_ADVANCED, orderState = {snapStart: 26655, snapEnd 26655, snapStarted true, prevSeq: 26656, prevSeqValid: true, errCount: 1}, event = Opcode DCP_SEQNO_ADVANCED, Status SUCCESS, Datatype 0, VBucket 72, Opaque 1286, VBuuid 237329969027338, Key <ud>()</ud>, Cas 0, Seqno 26655, RevSeqno 0, Flags 0, Expiry 0, LockTime 0, Nru 0, SnapstartSeq 0, SnapendSeq 0, SnapshotType 0, FailoverLog <nil>, Error <nil>, Ctime 1636446131040248105
|
/opt/couchbase/var/lib/couchbase/logs/projector.log:2021-11-09T00:22:11.192-08:00 [Fatal] ENDP[<-(172.23.97.217:9105,f4eb)<-127.0.0.1:8091 #MAINT_STREAM_TOPIC_799c07c811d6f05cef27c3dba8db6676] seq order violation for vb = bucket3:72, seq = 26655, command = 16, orderState = {snapStart: 26655, snapEnd 26655, snapStarted true, prevSeq: 26656, prevSeqValid: true, errCount: 1}, mutation = Docidx <ud>()</ud>, Seqno 26655, Ctime 0, Uuids [0], Commands [16]
|
/opt/couchbase/var/lib/couchbase/logs/projector.log:2021-11-09T00:22:11.680-08:00 [Fatal] DCPT[secidx:proj-bucket3-MAINT_STREAM_TOPIC_799c07c811d6f05cef27c3dba8db6676-3190590407978436790/3] seq order violation for vb = 111, seq = 25450, opcode = DCP_SEQNO_ADVANCED, orderState = {snapStart: 25450, snapEnd 25450, snapStarted true, prevSeq: 25451, prevSeqValid: true, errCount: 1}, event = Opcode DCP_SEQNO_ADVANCED, Status SUCCESS, Datatype 0, VBucket 111, Opaque 1286, VBuuid 81113905331924, Key <ud>()</ud>, Cas 0, Seqno 25450, RevSeqno 0, Flags 0, Expiry 0, LockTime 0, Nru 0, SnapstartSeq 0, SnapendSeq 0, SnapshotType 0, FailoverLog <nil>, Error <nil>, Ctime 1636446131680590923
|
/opt/couchbase/var/lib/couchbase/logs/projector.log:2021-11-09T00:22:11.779-08:00 [Fatal] ENDP[<-(172.23.97.217:9105,f4eb)<-127.0.0.1:8091 #MAINT_STREAM_TOPIC_799c07c811d6f05cef27c3dba8db6676] seq order violation for vb = bucket3:111, seq = 25450, command = 16, orderState = {snapStart: 25450, snapEnd 25450, snapStarted true, prevSeq: 25451, prevSeqValid: true, errCount: 1}, mutation = Docidx <ud>()</ud>, Seqno 25450, Ctime 0, Uuids [0], Commands [16]
|
This could be a regression since this issue was fixed in Cheshire Cat with MB-46466.
Note : OSO is enabled for GSI in this test.
Attachments
For Gerrit Dashboard: MB-49453 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
166077,5 | MB-49453: Don't generate a new snapshot when ending on a new checkpoint | master | kv_engine | Status: MERGED | +2 | +1 |