Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-43236

[Object Store] Use batch delete operations where possible

    XMLWordPrintable

Details

    • Task
    • Resolution: Done
    • Minor
    • 7.1.0
    • Cheshire-Cat
    • tools
    • None
    • 1

    Description

      What's the issue?
      We're currently performing bulk delete operations using a worker pool go goroutines, this is significantly faster than sequentially deleting data, however, the request overhead is still significant.

      Most object store providers implement batch operations for this where multiple operations can be batched into a single HTTP request, this will significantly improve the performance of the remove sub-command along with the '--purge' flag.

      AWS
      S3 exposes this functionality in the SDK already so we could take advantage of this immediately.

      Azure
      Although the HTTP API exposes these batch operations, the SDK does not implement this functionality, however, it does appear to be on the developers radar.

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-43236
          # Subject Branch Project Status CR V

          Activity

            People

              james.lee James Lee
              james.lee James Lee
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty