Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-43008

[System Test] : index building for a partitioned primary index is stuck

    XMLWordPrintable

    Details

      Description

      Build : 6.6.1-9198
      Test : -test tests/integration/test_allFeatures_madhatter_durability.yml -scope tests/integration/scope_Xattrs_Madhatter.yml
      Scale : 3
      Iteration : 2nd (Day 1)

      This seems to be the same issue as MB-37689 / MB-38232.

      Partitioned primary index on CUSTOMER bucket is stuck at 100%

      [2020-11-29T18:13:55-08:00, sequoiatools/cbq:8e2e15] -e=http://172.23.96.252:8093 -u=Administrator -p=password -script=create primary index on `CUSTOMER` partition by hash(rating,result,claim) using GSI with {"num_partition":4}
      

      This index is on nodes 172.23.104.69, 172.23.96.253 & 172.23.99.11

      Some errors in the indexer logs on 172.23.104.69 around the time of index creation:

      [root@cnt7-qe1 logs]# zgrep "2020-11-29T18:1" indexer.log* | grep CUSTOMER | grep "\[Error"
      indexer.log.2.gz:2020-11-29T18:14:25.162-08:00 [Error] KVSender::sendDelBucketsRequest Unexpected Error During Del Buckets Request Projector 172.23.104.5:9999 Topic INIT_STREAM_TOPIC_7f3c605a3eeeb5a18047b3e58f9a8373 Buckets [CUSTOMER]. Err genServer.closed
      indexer.log.2.gz:2020-11-29T18:14:25.162-08:00 [Error] KVSender::deleteBucketsFromStream INIT_STREAM CUSTOMER Error Received genServer.closed from 172.23.104.5:9999
      indexer.log.2.gz:2020-11-29T18:14:25.176-08:00 [Error] KVSender::sendDelBucketsRequest Unexpected Error During Del Buckets Request Projector 172.23.108.103:9999 Topic INIT_STREAM_TOPIC_7f3c605a3eeeb5a18047b3e58f9a8373 Buckets [CUSTOMER]. Err genServer.closed
      indexer.log.2.gz:2020-11-29T18:14:25.176-08:00 [Error] KVSender::deleteBucketsFromStream INIT_STREAM CUSTOMER Error Received genServer.closed from 172.23.108.103:9999
      indexer.log.2.gz:2020-11-29T18:14:25.180-08:00 [Error] KVSender::sendDelBucketsRequest Unexpected Error During Del Buckets Request Projector 172.23.96.148:9999 Topic INIT_STREAM_TOPIC_7f3c605a3eeeb5a18047b3e58f9a8373 Buckets [CUSTOMER]. Err genServer.closed
      indexer.log.2.gz:2020-11-29T18:14:25.181-08:00 [Error] KVSender::deleteBucketsFromStream INIT_STREAM CUSTOMER Error Received genServer.closed from 172.23.96.148:9999
      indexer.log.2.gz:2020-11-29T18:14:25.186-08:00 [Error] KVSender::sendDelBucketsRequest Unexpected Error During Del Buckets Request Projector 172.23.97.119:9999 Topic INIT_STREAM_TOPIC_7f3c605a3eeeb5a18047b3e58f9a8373 Buckets [CUSTOMER]. Err genServer.closed
      indexer.log.2.gz:2020-11-29T18:14:25.186-08:00 [Error] KVSender::deleteBucketsFromStream INIT_STREAM CUSTOMER Error Received genServer.closed from 172.23.97.119:9999
      indexer.log.2.gz:2020-11-29T18:14:25.186-08:00 [Error] KVSender::sendDelBucketsRequest Unexpected Error During Del Buckets Request Projector 172.23.97.121:9999 Topic INIT_STREAM_TOPIC_7f3c605a3eeeb5a18047b3e58f9a8373 Buckets [CUSTOMER]. Err projector.topicMissing
      indexer.log.2.gz:2020-11-29T18:14:25.186-08:00 [Error] KVSender::deleteBucketsFromStream INIT_STREAM CUSTOMER Error Received projector.topicMissing from 172.23.97.121:9999
      indexer.log.2.gz:2020-11-29T18:14:25.199-08:00 [Error] KVSender::sendDelBucketsRequest Unexpected Error During Del Buckets Request Projector 172.23.99.20:9999 Topic INIT_STREAM_TOPIC_7f3c605a3eeeb5a18047b3e58f9a8373 Buckets [CUSTOMER]. Err genServer.closed
      indexer.log.2.gz:2020-11-29T18:14:25.199-08:00 [Error] KVSender::deleteBucketsFromStream INIT_STREAM CUSTOMER Error Received genServer.closed from 172.23.99.20:9999
      indexer.log.2.gz:2020-11-29T18:14:25.204-08:00 [Error] KVSender::sendDelBucketsRequest Unexpected Error During Del Buckets Request Projector 172.23.99.21:9999 Topic INIT_STREAM_TOPIC_7f3c605a3eeeb5a18047b3e58f9a8373 Buckets [CUSTOMER]. Err projector.topicMissing
      indexer.log.2.gz:2020-11-29T18:14:25.204-08:00 [Error] KVSender::deleteBucketsFromStream INIT_STREAM CUSTOMER Error Received projector.topicMissing from 172.23.99.21:9999
      indexer.log.2.gz:2020-11-29T18:14:25.210-08:00 [Error] KVSender::sendDelBucketsRequest Unexpected Error During Del Buckets Request Projector 172.23.99.25:9999 Topic INIT_STREAM_TOPIC_7f3c605a3eeeb5a18047b3e58f9a8373 Buckets [CUSTOMER]. Err genServer.closed
      indexer.log.2.gz:2020-11-29T18:14:25.210-08:00 [Error] KVSender::deleteBucketsFromStream INIT_STREAM CUSTOMER Error Received genServer.closed from 172.23.99.25:9999
      

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-7.0.0-4156 contains indexing commit b90f5dd with commit message:
            MB-43008: Introduce logging for sequence number violation in projector

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-4156 contains indexing commit b90f5dd with commit message: MB-43008 : Introduce logging for sequence number violation in projector
            Hide
            mihir.kamdar Mihir Kamdar added a comment -

            Closing as per Deep's comment w.r.t. the defensive fix, on 6.6.1-9213

            Show
            mihir.kamdar Mihir Kamdar added a comment - Closing as per Deep's comment w.r.t. the defensive fix, on 6.6.1-9213
            Hide
            deepkaran.salooja Deepkaran Salooja added a comment - - edited

            Centos1 cluster hit into the issue on 172.23.99.11:
             

            2020-12-12T12:06:41.850-08:00 [Warn] Indexer::MutationQueue Dequeue Aborted For Seqno 252528 Bucket default Vbucket 536. Last Dequeue 252493 Head Seqno 252597.2020-12-12T12:06:41.854-08:00 [Info] Timekeeper::processFlushAbort Flush Abort Received MAINT_STREAM default
            FlushTs bucket: default, vbuckets: 1024 Crc64: 5001023028077095860 snapType INMEM_SNAP -2020-12-12T12:06:41.857-08:00 [Info] Timekeeper::processFlushAbort MAINT_STREAM default 3 Generate InitPrepRecovery 
            

            With the fix, system was able to recover.

            Show
            deepkaran.salooja Deepkaran Salooja added a comment - - edited Centos1 cluster hit into the issue on 172.23.99.11 :   2020-12-12T12:06:41.850-08:00 [Warn] Indexer::MutationQueue Dequeue Aborted For Seqno 252528 Bucket default Vbucket 536. Last Dequeue 252493 Head Seqno 252597.2020-12-12T12:06:41.854-08:00 [Info] Timekeeper::processFlushAbort Flush Abort Received MAINT_STREAM default FlushTs bucket: default, vbuckets: 1024 Crc64: 5001023028077095860 snapType INMEM_SNAP -2020-12-12T12:06:41.857-08:00 [Info] Timekeeper::processFlushAbort MAINT_STREAM default 3 Generate InitPrepRecovery With the fix, system was able to recover.
            Hide
            mihir.kamdar Mihir Kamdar added a comment -

            Latest logs from 6.6.1-9213 :

            Centos1 cluster (172.23.108.103)
            Index Nodes : 172.23.104.137, 172.23.104.155, 172.23.104.70, 172.23.96.253, 172.23.96.95, 172.23.99.11
            Logs:
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.137.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.155.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.157.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.5.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.67.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.69.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.70.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.106.188.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.108.103.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.120.245.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.121.3.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.123.27.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.123.28.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.148.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.251.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.252.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.253.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.95.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.97.119.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.97.121.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.97.122.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.97.239.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.98.135.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.99.11.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.99.20.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.99.21.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.99.25.zip

            Centos2 Cluster (172.23.97.74)
            Index Nodes : 172.23.106.134, 172.23.120.58, 172.23.120.81, 172.23.96.243, 172.23.97.105, 172.23.97.110
            Logs :
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.106.134.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.58.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.74.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.75.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.77.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.81.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.86.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.121.77.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.123.25.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.123.26.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.123.31.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.123.33.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.122.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.14.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.18.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.243.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.254.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.48.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.105.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.110.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.112.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.148.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.149.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.150.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.151.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.74.zip

            Show
            mihir.kamdar Mihir Kamdar added a comment - Latest logs from 6.6.1-9213 : Centos1 cluster (172.23.108.103) Index Nodes : 172.23.104.137, 172.23.104.155, 172.23.104.70, 172.23.96.253, 172.23.96.95, 172.23.99.11 Logs: url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.137.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.155.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.157.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.5.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.67.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.69.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.104.70.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.106.188.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.108.103.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.120.245.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.121.3.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.123.27.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.123.28.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.148.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.251.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.252.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.253.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.96.95.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.97.119.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.97.121.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.97.122.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.97.239.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.98.135.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.99.11.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.99.20.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.99.21.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607809734/collectinfo-2020-12-12T214857-ns_1%40172.23.99.25.zip Centos2 Cluster (172.23.97.74) Index Nodes : 172.23.106.134, 172.23.120.58, 172.23.120.81, 172.23.96.243, 172.23.97.105, 172.23.97.110 Logs : url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.106.134.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.58.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.74.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.75.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.77.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.81.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.120.86.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.121.77.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.123.25.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.123.26.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.123.31.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.123.33.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.122.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.14.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.18.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.243.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.254.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.96.48.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.105.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.110.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.112.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.148.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.149.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.150.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.151.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1607805003/collectinfo-2020-12-12T203004-ns_1%40172.23.97.74.zip
            Hide
            deepkaran.salooja Deepkaran Salooja added a comment -

            Defensive fix has been merged. Opened MB-43259 to track the underlying issue.

            Show
            deepkaran.salooja Deepkaran Salooja added a comment - Defensive fix has been merged. Opened  MB-43259 to track the underlying issue.

              People

              Assignee:
              mihir.kamdar Mihir Kamdar
              Reporter:
              mihir.kamdar Mihir Kamdar
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                    PagerDuty