Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47849

Eventing does not clean up / undeploy functions when bucket deleted

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 7.1.0
    • None
    • eventing
    • None
    • Untriaged
    • 1
    • Unknown

    Description

      While working on the eventing management API, I've been cycling through lots of "bucket creation, do stuff, delete bucket". After some time I noticed that my tests were behaving weird.

      I jumped over to the UI and saw this:

      These all seem to be functions which are currently deploying (so I cannot even delete them), but all their source and meta buckets are already gone. In this case the bucket is the same, but the collection is different.

      If you click edit on one, you can see that the bucket is blank:

      All those buckets are long gone, and I have a suspicion that they failed to be cleaned up while they were deploying and now they are hung in that state.

      It's not a UI bug, since a direct curl shows them:

      $ curl http://127.0.0.1:8096/api/v1/functions -u Administrator:password
      [
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "abd24c48-b5bf-4549-98f0-c802f5be8d9f",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "abd24c48-b5bf-4549-98f0-c802f5be8d9f",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 2644129492,
        "function_instance_id": "APv3C2",
        "appname": "f589309b-81ad-4963-8c9b-a19d9d54060b",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": true,
         "language_compatibility": "6.5.0",
         "processing_status": true
        }
       },
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "1b3b8a25-d8c0-49cb-bab8-ac25f4c228a5",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "1b3b8a25-d8c0-49cb-bab8-ac25f4c228a5",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 425521506,
        "function_instance_id": "OZte*1",
        "appname": "32a494da-6ce1-472c-af9c-d031ab2d0ccc",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": false,
         "language_compatibility": "6.5.0",
         "processing_status": false
        }
       },
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "8316ae07-ab7b-4986-a3af-8b9205b8dcdf",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "8316ae07-ab7b-4986-a3af-8b9205b8dcdf",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 2786231355,
        "function_instance_id": "1Hw*k2",
        "appname": "f6756f7c-e97c-4a0f-a77f-0bb0ee899c3b",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": true,
         "language_compatibility": "6.5.0",
         "processing_status": true
        }
       },
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "1ea83730-649a-4a7c-a084-8f5100b0f82a",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "1ea83730-649a-4a7c-a084-8f5100b0f82a",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 1566367657,
        "function_instance_id": "mMme33",
        "appname": "ca67ca95-7418-4c42-a906-e2c3ee11fb36",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": true,
         "language_compatibility": "6.5.0",
         "processing_status": true
        }
       },
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "26a4547a-e2b3-41f3-aeb5-8947fd758dba",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "26a4547a-e2b3-41f3-aeb5-8947fd758dba",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 2954262205,
        "function_instance_id": "jYQmR",
        "appname": "8f0d414b-0022-43ec-a825-b3200afd18a4",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": true,
         "language_compatibility": "6.5.0",
         "processing_status": true
        }
       },
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "6b681293-dd1e-4236-8f29-a80a409f711e",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "6b681293-dd1e-4236-8f29-a80a409f711e",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 2852908417,
        "function_instance_id": "b\u0026tpN2",
        "appname": "0070cd68-ccca-4b97-94cc-d61c8ac54402",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": true,
         "language_compatibility": "6.5.0",
         "processing_status": true
        }
       },
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "49302cc8-d34a-497e-82cf-3c7c9953215f",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "49302cc8-d34a-497e-82cf-3c7c9953215f",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 202027402,
        "function_instance_id": "zTqy8",
        "appname": "230c40ac-0a91-43e0-9807-2accb9ee367e",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": true,
         "language_compatibility": "6.5.0",
         "processing_status": true
        }
       },
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "74f2ad14-e8b4-45f8-905d-5385ecc37b57",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "74f2ad14-e8b4-45f8-905d-5385ecc37b57",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 2902850457,
        "function_instance_id": "X4Xdc",
        "appname": "a4c6295a-28a8-4573-a9ca-9b52866ef7aa",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": true,
         "language_compatibility": "6.5.0",
         "processing_status": true
        }
       },
       {
        "appcode": "function OnUpdate(doc, meta) {}",
        "depcfg": {
         "source_bucket": "c2c24cd6-4dde-453c-bbf5-8f1ca5d5fd79",
         "source_scope": "eventing",
         "source_collection": "source",
         "metadata_bucket": "c2c24cd6-4dde-453c-bbf5-8f1ca5d5fd79",
         "metadata_scope": "eventing",
         "metadata_collection": "meta"
        },
        "version": "evt-7.0.0-5302-ee",
        "enforce_schema": false,
        "handleruuid": 886710955,
        "function_instance_id": "BdyYV3",
        "appname": "9e9e496d-ea6a-474f-9ec8-ae2e241131d9",
        "settings": {
         "dcp_stream_boundary": "everything",
         "deployment_status": true,
         "language_compatibility": "6.5.0",
         "processing_status": true
        }
       }
      ]
      

      I've attached my eventing.log, there is maybe lots going on but look for entries from today (2021-08-10)

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            sujay.gad Sujay Gad
            daschl Michael Nitschinger
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty