Details
-
Bug
-
Resolution: Fixed
-
Test Blocker
-
Columnar 1.0.0, 7.6.2, 7.2.6
-
1.0.0-2179
-
Untriaged
-
ingestion_rollback_restart → http://supportal.couchbase.com/snapshot/8eb074f1689b304bb8e2d56ac3b72a80::1
-
0
-
Unknown
-
Analytics Sprint 45
Description
- Create 5 remote collections each having 1B items:
creating remote collections on couchbase: remote_jRNk8_volCollection_0_ozzemcreating remote collections on couchbase: remote_jRNk8_volCollection_1_jfqjacreating remote collections on couchbase: remote_jRNk8_volCollection_2_enbkncreating remote collections on couchbase: remote_jRNk8_volCollection_3_lnxrgcreating remote collections on couchbase: remote_jRNk8_volCollection_4_lwfpo - Wait for ingestion completion
- Create 5 new remote collections:
creating remote collections on couchbase: remote_jRNk8_volCollection_0_sdsnucreating remote collections on couchbase: remote_jRNk8_volCollection_1_mvhsicreating remote collections on couchbase: remote_jRNk8_volCollection_2_zmffucreating remote collections on couchbase: remote_jRNk8_volCollection_3_memkacreating remote collections on couchbase: remote_jRNk8_volCollection_4_tiprk - While ingestion in happening perform scaling operations:
Scaling OUT operation: 8 -> 4Scaling OUT operation: 4 -> 8Scaling IN operation: 8 -> 4Scaling IN operation: 4 -> 2 - Repeat the above step. Create 5 new collections and perform scaling:
creating remote collections on couchbase: remote_jRNk8_volCollection_0_jsosscreating remote collections on couchbase: remote_jRNk8_volCollection_1_jjaeocreating remote collections on couchbase: remote_jRNk8_volCollection_2_acgnzcreating remote collections on couchbase: remote_jRNk8_volCollection_3_gdglpcreating remote collections on couchbase: remote_jRNk8_volCollection_4_oosjpScaling OUT operation: 2 ->4Scaling OUT operation: 4 ->8Scaling IN operation: 8 -> 4 - Observe that the count of the latest created collections is oscillating between 0 and xxxxx multiple times as shown below:
creating remote collections on couchbase: remote_jRNk8_volCollection_3_gdglpdataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 381, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 3888, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 11808, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 28433, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 39626, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 44117, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 48213, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 50908, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 55716, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 72141, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 83147, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 102339, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 114803, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 116197, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 124029, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 136677, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 143276, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 152021, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 161371, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 169915, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 177040, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 177226, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 177226, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 177226, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 177516, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 184055, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 187995, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189317, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 189842, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 208866, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 640403, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 1723207, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 2149359, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 2278452, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 2317418, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 2317426, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 2318841, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 2460492, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 2729224, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 3550677, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4273013, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395030, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4395768, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4405040, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4459897, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4478393, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 3953422, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 2239175, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 1959440, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 104804, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4650, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15429, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 1016062, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 3176574, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 3186159, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 3240672, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 4225875, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 6370203, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 6395976, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 7127288, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 8434718, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 8737790, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 8808781, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 9635511, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 10872213, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 11105051, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 12304664, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 13382260, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 13697160, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 14699152, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15570704, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15708843, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15732298, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15787549, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15787549, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15787549, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15787549, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15787549, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15787549, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15787549, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15541703, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 8017537, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 255397, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 249696, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 249696, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 249696, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 249696, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 249696, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 415886, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 415886, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 415886, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 155396, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 15226, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 106353, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 108710, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 183425, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 183425, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 183425, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 146984, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 54036, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000dataset: remote_jRNk8_volCollection_3_gdglp, status: SUCCESS, actual count: 0, expected count: 1000000000
Not sure if this is related but it has been see in the logs:
analytics_info.log.3.gz:2024-06-26T08:12:19.110+00:00 ERRO CBAS.active.ActiveNotificationHandler [ActiveNotificationHandler] entity not found for event JID:0.8385, EntityId:remote_jRNk8/default0_QNK77, Kind:PARTITION_EVENT-GENERIC_EVENT(IdleThresholdEvent{nodeId='f42617c1f6299cf6ca05e976b28d1cbf'})
|
analytics_info.log.3.gz:2024-06-26T08:12:19.138+00:00 ERRO CBAS.active.ActiveNotificationHandler [ActiveNotificationHandler] entity not found for event JID:0.8385, EntityId:remote_jRNk8/default0_QNK77, Kind:PARTITION_EVENT-GENERIC_EVENT(IdleThresholdEvent{nodeId='e73434f447251ae2804c37ddd233d4c0'})
|
analytics_info.log.3.gz:2024-06-26T08:12:19.171+00:00 ERRO CBAS.active.ActiveNotificationHandler [ActiveNotificationHandler] entity not found for event JID:0.8385, EntityId:remote_jRNk8/default0_QNK77, Kind:PARTITION_EVENT-GENERIC_EVENT(IdleThresholdEvent{nodeId='cffe031812d4b6591614138879fec2ac'})
|
analytics_info.log.3.gz:2024-06-26T08:12:24.044+00:00 ERRO CBAS.active.ActiveNotificationHandler [ActiveNotificationHandler] entity not found for event JID:0.8387, EntityId:remote_jRNk8/default0_QNK77, Kind:PARTITION_EVENT-GENERIC_EVENT(IdleThresholdEvent{nodeId='f42617c1f6299cf6ca05e976b28d1cbf'})
|
analytics_info.log.3.gz:2024-06-26T08:12:24.056+00:00 ERRO CBAS.active.ActiveNotificationHandler [ActiveNotificationHandler] entity not found for event JID:0.8387, EntityId:remote_jRNk8/default0_QNK77, Kind:PARTITION_EVENT-GENERIC_EVENT(IdleThresholdEvent{nodeId='e73434f447251ae2804c37ddd233d4c0'})
|
analytics_info.log.3.gz:2024-06-26T08:12:24.067+00:00 ERRO CBAS.active.ActiveNotificationHandler [ActiveNotificationHandler] entity not found for event JID:0.8387, EntityId:remote_jRNk8/default0_QNK77, Kind:PARTITION_EVENT-GENERIC_EVENT(IdleThresholdEvent{nodeId='cffe031812d4b6591614138879fec2ac'})
|
analytics_info.log.3.gz:2024-06-26T08:12:49.844+00:00 ERRO CBAS.active.ActiveNotificationHandler [ActiveNotificationHandler] entity not found for event JID:0.8763, EntityId:remote_jRNk8/default0_QNK77, Kind:PARTITION_EVENT-GENERIC_EVENT(IdleThresholdEvent{nodeId='e73434f447251ae2804c37ddd233d4c0'})
|
