Details
-
Bug
-
Resolution: Not a Bug
-
Critical
-
Columnar 1.0.0
-
Columnar Edition 1.0.0 build 2160
-
Untriaged
-
0
-
Unknown
Description
During the system test run, it was seen that not all the documents got ingested into the linked datasets via Amazon MSK. There are 49 linked datasets and in 18 of them, the total doc count is lower than expected ( 7074854) . The actual document count has to be 9999999. All the datasets are ingesting from the same Mongo collection ( systemTest.systemTestCollection).
Doc count for each of the linked dataset -
Item count in collection Database0AyneQAlx.scope0eIlfAGqc.LinkedDatasetZrMiPgeqLK = 9999999.000000 |
Item count in collection Database0AyneQAlx.scope0eIlfAGqc.LinkedDatasetgjbPBgdKEp = 9999999.000000 |
Item count in collection Database0AyneQAlx.scope1cwapeXPK.LinkedDatasetdwpplmmKbG = 9999999.000000 |
Item count in collection Database10cnWETRAy.scope0WaiABatF.LinkedDatasetRLJgoxdIdR = 9999999.000000 |
Item count in collection Database10cnWETRAy.scope0WaiABatF.LinkedDatasetjVYprzKtUH = 9999999.000000 |
Item count in collection Database11fiadwqYc.scope0ETflKXbk.LinkedDatasetgRFQCGUOpO = 7074854.000000 |
Item count in collection Database11fiadwqYc.scope1xNBtDcyy.LinkedDatasetyhQzpXnUcQ = 9999999.000000 |
Item count in collection Database12rJdIQPpJ.scope1PXxOfWpd.LinkedDatasetGKgMQwhwIj = 9999999.000000 |
Item count in collection Database12rJdIQPpJ.scope1PXxOfWpd.LinkedDatasetgOEhHdUFzc = 7074854.000000 |
Item count in collection Database13fqNHkEWO.scope0lljHQyuy.LinkedDatasetEgjzjQtPTN = 7074854.000000 |
Item count in collection Database13fqNHkEWO.scope0lljHQyuy.LinkedDatasetNRtGbqKDiZ = 9999999.000000 |
Item count in collection Database13fqNHkEWO.scope0lljHQyuy.LinkedDatasetWvJvKFFPZh = 9999999.000000 |
Item count in collection Database14jIsNqYqh.scope1FejwxDtu.LinkedDatasetUlizpJqZfr = 7074854.000000 |
Item count in collection Database14jIsNqYqh.scope1FejwxDtu.LinkedDatasetVUalZvZqIe = 9999999.000000 |
Item count in collection Database15PIqJMYmg.scope0nOjAWSjI.LinkedDatasetZnnsMPjvgu = 7074854.000000 |
Item count in collection Database15PIqJMYmg.scope0nOjAWSjI.LinkedDatasethQUCbkJaxD = 7074854.000000 |
Item count in collection Database16PUGCzSjX.scope0FHCkxPNy.LinkedDatasetuGetpaPygX = 9999999.000000 |
Item count in collection Database16PUGCzSjX.scope1gJRWJWId.LinkedDatasetpLnaDbUNwz = 9999999.000000 |
Item count in collection Database18NsAFRcyW.scope1nJMYsEzW.LinkedDatasetLVgMgfZMvj = 9999999.000000 |
Item count in collection Database19gSCPunOB.scope1gsFmrycX.LinkedDatasetZQTJGKmylf = 9999999.000000 |
Item count in collection Database1XgUOHVKW.scope0mYlzzvIc.LinkedDatasetNTZDkRYjAG = 7074854.000000 |
Item count in collection Database1XgUOHVKW.scope0mYlzzvIc.LinkedDatasetkVQnyoIxLI = 9999999.000000 |
Item count in collection Database1XgUOHVKW.scope1OshxVvUQ.LinkedDatasetsTTmYNRNCR = 9999999.000000 |
Item count in collection Database20hrjmhiIa.scope1GqCZAPmF.LinkedDatasetXYxxTjsaHX = 9999999.000000 |
Item count in collection Database21PbwVHqnV.scope1TisyjaHc.LinkedDatasetyXTobQuDmo = 7074854.000000 |
Item count in collection Database23MjTJjiIb.scope0wqPZVHGA.LinkedDatasetMlLlAJABJC = 9999999.000000 |
Item count in collection Database23MjTJjiIb.scope0wqPZVHGA.LinkedDatasetzidbtJLuxj = 7074854.000000 |
Item count in collection Database24UJhjVoCT.scope0UwRKdMTH.LinkedDataseticpkSPFCQG = 9999999.000000 |
Item count in collection Database24UJhjVoCT.scope1XASCjLQd.LinkedDatasetOHmOQRHqsM = 7074854.000000 |
Item count in collection Database2reFJXiTT.scope1tBUCoksu.LinkedDatasetPdozpuiUjK = 7074854.000000 |
Item count in collection Database2reFJXiTT.scope1tBUCoksu.LinkedDatasetdYigxwYvYz = 9999999.000000 |
Item count in collection Database3WAldLlqm.scope0pQiKdNSX.LinkedDatasetMxnAxGpxbp = 9999999.000000 |
Item count in collection Database3WAldLlqm.scope0pQiKdNSX.LinkedDatasetUqDJmKWPyS = 9999999.000000 |
Item count in collection Database3WAldLlqm.scope1EOSsgRsr.LinkedDatasetmvxNPPlpPH = 7074854.000000 |
Item count in collection Database3WAldLlqm.scope1EOSsgRsr.LinkedDatasetsxiNyEmjdE = 9999999.000000 |
Item count in collection Database4LPfkuqMp.scope1WvPJtGSz.LinkedDatasetLamXqZcLfL = 9999999.000000 |
Item count in collection Database5xOYIAltd.scope1oPGhbmEx.LinkedDatasetAInxnUwwyP = 9999999.000000 |
Item count in collection Database6IWLYPKiM.scope1WtwCJTLX.LinkedDatasetNJtTyfAezH = 7074854.000000 |
Item count in collection Database6IWLYPKiM.scope1WtwCJTLX.LinkedDatasetROnQjYoGuM = 7074854.000000 |
Item count in collection Database6IWLYPKiM.scope1WtwCJTLX.LinkedDatasetXyWUdTXldF = 9999999.000000 |
Item count in collection Database8IGIzJVVR.scope0CGHHDyxO.LinkedDatasetfWcDTJVOTW = 7074854.000000 |
Item count in collection Database8IGIzJVVR.scope0CGHHDyxO.LinkedDatasetjxEStypYfb = 9999999.000000 |
Item count in collection Database8IGIzJVVR.scope0CGHHDyxO.LinkedDatasetqDxMoDrxeV = 9999999.000000 |
Item count in collection Database8IGIzJVVR.scope0CGHHDyxO.LinkedDatasetwuScVCRTSg = 7074854.000000 |
Item count in collection Database8IGIzJVVR.scope1HBEmfwmR.LinkedDatasetHbKwgtsubS = 9999999.000000 |
Item count in collection Database8IGIzJVVR.scope1HBEmfwmR.LinkedDatasetjUGlPKZNar = 9999999.000000 |
Item count in collection Database8IGIzJVVR.scope1HBEmfwmR.LinkedDatasetlkvFVwmXOz = 9999999.000000 |
Item count in collection Database9gvdDkHTh.scope0tUrwSHfV.LinkedDatasetqWgSvtxUIy = 7074854.000000 |
Item count in collection Database9gvdDkHTh.scope1BdClCpoT.LinkedDatasetjykgXvUjZM = 7074854.000000 |
Some of the collections that show a lower doc count -
Database12rJdIQPpJ.scope1PXxOfWpd.LinkedDatasetgOEhHdUFzc
|
Database11fiadwqYc.scope0ETflKXbk.LinkedDatasetgRFQCGUOpO
|
Database15PIqJMYmg.scope0nOjAWSjI.LinkedDatasetZnnsMPjvgu
|
Database15PIqJMYmg.scope0nOjAWSjI.LinkedDatasethQUCbkJaxD
|
Strangely, for all the collections showing a lower count, it shows the same count (7074854.000000).
cbcollect ->
https://cb-engineering.s3.amazonaws.com/SysTestColumnar18Jun/collectinfo-2024-06-18T183829-ns_1%40svc-da-node-001.45yxxjbrevilttq.sandbox.nonprod-project-avengers.com.zip
https://cb-engineering.s3.amazonaws.com/SysTestColumnar18Jun/collectinfo-2024-06-18T183829-ns_1%40svc-da-node-002.45yxxjbrevilttq.sandbox.nonprod-project-avengers.com.zip
https://cb-engineering.s3.amazonaws.com/SysTestColumnar18Jun/collectinfo-2024-06-18T183829-ns_1%40svc-da-node-003.45yxxjbrevilttq.sandbox.nonprod-project-avengers.com.zip
https://cb-engineering.s3.amazonaws.com/SysTestColumnar18Jun/collectinfo-2024-06-18T183829-ns_1%40svc-da-node-004.45yxxjbrevilttq.sandbox.nonprod-project-avengers.com.zip