Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-52440

Eventing: Handler stuck in undeploying state

    XMLWordPrintable

Details

    • Bug
    • Resolution: User Error
    • Critical
    • 7.1.2
    • 7.1.1
    • storage-engine
    • 7.1.1-3105

    Description

      QE TEST

      ./testrunner -i /tmp/testexec.60106.ini -p get-cbcollect-info=True,reset_services=False,java_sdk_client=True,bucket_storage=magma,GROUP=magma_40dgm_set4 -t eventing.eventing_sanity.EventingSanity.test_expired_mutation,nodes_init=4,services_init=kv-kv-eventing-index:n1ql,dataset=default,groups=simple,reset_services=False,bucket_storage=magma,number_of_documents=1250000,magma_bucket_quota=512,eviction_policy=fullEviction,dgm_run=True,batch_size=100000,non_default_collection=True,java_sdk_client=True,GROUP=magma_40dgm_set4,get-cbcollect-info=True

      ISSUE
      Eventing handler is stuck in undeploying state.

      OBSERVATION
      From eventing.log it appears cleanup of artefacts from metadata bucket took over 2 minutes. There are no other error messages in logs during undeployment timeframe.

      2022-06-02T13:41:38.036-07:00 [Info] Producer::cleanupMetadataImpl [src_bucket/src_bucket/Function_995760096_test_expired_mutation:0:id_7] Started up dcpfeed to cleanup artifacts from metadata bucket: metadata
      		 
      2022-06-02T13:41:38.037-07:00 [Info] Producer::cleanupMetadataImpl [src_bucket/src_bucket/Function_995760096_test_expired_mutation:0:id_3] Started up dcpfeed to cleanup artifacts from metadata bucket: metadata
      		 
      2022-06-02T13:44:08.124-07:00 [Info] Producer::cleanupMetadataImpl [src_bucket/src_bucket/Function_995760096_test_expired_mutation:0:id_1] Exiting metadata cleanup routine, mutations till high vb seqnos received

      NOTE
      This issue is not reproducible consistently, so far have run into it only once.

      Attachments

        1. compactwritebytes_persec.png
          compactwritebytes_persec.png
          243 kB
        2. diskwrite_drain_rate.png
          diskwrite_drain_rate.png
          351 kB
        3. diskwritequeue.png
          diskwritequeue.png
          131 kB
        4. test.log
          142 kB
        5. writebytes_persec.png
          writebytes_persec.png
          163 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            sujay.gad Sujay Gad
            sujay.gad Sujay Gad
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty