Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-33750

Ensure that when a new node picks up orchetratorship, the lease_acquirer process on the old orchestrator is shutdown

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 6.0.2
    • 5.5.0, 5.5.1, 5.5.2, 5.5.3
    • ns_server
    • Untriaged
    • Unknown

    Description

      We have seen a couple of cases in which a rogue lease acquirer process is left running on a node that used to be the orchestrator. This can happen via a race condition that's exposed if a supervisor that is shutting down normally, is for some reason brutally shutdown leaving an orphan child process.

      The rogue lease acquirer interferes with the operation of the orchestrator causing, for instance, rebalance to fail.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-33853 [5.5.4] - Ensure that when a new node picks up orchetratorship, the lease_acquirer process on the old orchestrator is shutdown

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          For Gerrit Dashboard: MB-33750
          # Subject Branch Project Status CR V
          107755,1 [bp] Remove the timeout logic from mb_master:shutdown_master_sup. master ns_server Status: ABANDONED 0 0
          107756,1 [bp] Use shutdown_master_sup in mb_master:terminate. master ns_server Status: ABANDONED 0 0
          107757,1 [bp] Split leader_services_sup into three different modules. master ns_server Status: ABANDONED 0 0
          107758,1 [bp] Start leader_events outside of leader_service_sup. master ns_server Status: ABANDONED 0 0
          107759,1 [bp] Make sure ticks are only accepted from one node at a time. master ns_server Status: ABANDONED 0 0
          107760,4 [bp] Remove the timeout logic from mb_master:shutdown_master_sup. alice ns_server Status: MERGED +2 +1
          107761,4 [bp] Use shutdown_master_sup in mb_master:terminate. alice ns_server Status: MERGED +2 +1
          107762,1 [bp] Split leader_services_sup into three different modules. alice ns_server Status: ABANDONED 0 0
          107763,1 [bp] Start leader_events outside of leader_service_sup. alice ns_server Status: ABANDONED 0 0
          107764,1 [bp] Make sure ticks are only accepted from one node at a time. alice ns_server Status: ABANDONED 0 -1
          108003,6 MB-33750 Try working around the bug upon taking over mastership. alice ns_server Status: MERGED +2 +1
          108465,1 Merge remote-tracking branch 'couchbase/alice' into merge master ns_server Status: MERGED +2 +1
          188618,13 MB-55576: Remove MB-33750 workaround master ns_server Status: MERGED +2 +1

          Activity

            People

              Aliaksey Artamonau Aliaksey Artamonau (Inactive)
              dfinlay Dave Finlay
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty