Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-52440

Eventing: Handler stuck in undeploying state

    XMLWordPrintable

Details

    • Bug
    • Resolution: User Error
    • Critical
    • 7.1.2
    • 7.1.1
    • storage-engine
    • 7.1.1-3105

    Description

      QE TEST

      ./testrunner -i /tmp/testexec.60106.ini -p get-cbcollect-info=True,reset_services=False,java_sdk_client=True,bucket_storage=magma,GROUP=magma_40dgm_set4 -t eventing.eventing_sanity.EventingSanity.test_expired_mutation,nodes_init=4,services_init=kv-kv-eventing-index:n1ql,dataset=default,groups=simple,reset_services=False,bucket_storage=magma,number_of_documents=1250000,magma_bucket_quota=512,eviction_policy=fullEviction,dgm_run=True,batch_size=100000,non_default_collection=True,java_sdk_client=True,GROUP=magma_40dgm_set4,get-cbcollect-info=True

      ISSUE
      Eventing handler is stuck in undeploying state.

      OBSERVATION
      From eventing.log it appears cleanup of artefacts from metadata bucket took over 2 minutes. There are no other error messages in logs during undeployment timeframe.

      2022-06-02T13:41:38.036-07:00 [Info] Producer::cleanupMetadataImpl [src_bucket/src_bucket/Function_995760096_test_expired_mutation:0:id_7] Started up dcpfeed to cleanup artifacts from metadata bucket: metadata
      		 
      2022-06-02T13:41:38.037-07:00 [Info] Producer::cleanupMetadataImpl [src_bucket/src_bucket/Function_995760096_test_expired_mutation:0:id_3] Started up dcpfeed to cleanup artifacts from metadata bucket: metadata
      		 
      2022-06-02T13:44:08.124-07:00 [Info] Producer::cleanupMetadataImpl [src_bucket/src_bucket/Function_995760096_test_expired_mutation:0:id_1] Exiting metadata cleanup routine, mutations till high vb seqnos received

      NOTE
      This issue is not reproducible consistently, so far have run into it only once.

      Attachments

        1. test.log
          142 kB
        2. compactwritebytes_persec.png
          compactwritebytes_persec.png
          243 kB
        3. diskwritequeue.png
          diskwritequeue.png
          131 kB
        4. writebytes_persec.png
          writebytes_persec.png
          163 kB
        5. diskwrite_drain_rate.png
          diskwrite_drain_rate.png
          351 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            sujay.gad Sujay Gad
            sujay.gad Sujay Gad
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty