Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48474

Backup Service: when choosing a cbbs node to run a job on, does not look at whether the node already has a running job

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 7.1.0
    • 7.0.0, 7.0.1, 7.0.2
    • tools
    • None
    • Untriaged
    • 1
    • Unknown

    Description

      In a cluster, I have 2 nodes running the Backup Service, and I have two active backup repositories, each doing a backup of a bucket on the cluster – repo1 backs up bucket1 and repo2 backs up bucket2.  (The backup for both bucket1 and bucket2 are being done to S3, but the archive location type – S3 or filesystem/NFS – does not matter.)

      Each backup – repo1 and repo2 – takes a few minutes to run.  Regardless of whether I manually start the backups (one right after the other) – or the scheduled backups are running, both repo1 and repo2 backups run on the same cbbs node, serially, with the second backup waiting until the running backup finishes (status says "waiting to run backup"), instead of running on the other cbbs node, which isn't doing anything.   Spoke to James Lee who verified that the algorithm for dispatching the jobs to the cbbs nodes looks only at the number of jobs queued on the cbbs nodes (both empty when dispatching the second job in my scenario) and does not look at whether the cbbs node has a currently running job (which could take a while to complete).

      Note:  In my repro, I see that the status for one repo shows "running backup", and the status for the other repo shows "waiting to run backup" in the UI.

      Attachments

        For Gerrit Dashboard: MB-48474
        # Subject Branch Project Status CR V

        Activity

          People

            joe.mitchelljones Joe Mitchell Jones
            hyun-ju.vega Hyun-Ju Vega
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty