Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-55579

No downtime migration from couchstore to magma buckets

    XMLWordPrintable

Details

    • 0

    Description

      With magma bucket being the preferred storage engine going forward and with couchstore eventually being deprecated, we need to provide a way for customer to migrate from couchstore buckets to magma buckets without downtime.

      There are several solutions to this problem.
      One approach would be for KV and magma to provide a seemless method to move files from couchstore to magma. There are multiple solutions possible with this type of approach, however, the risk of regression is high and the task is very complex and littered with potential races.

      Another perhaps simpler solution is for ns_server to aid in the migration. The idea is that ns_server allows a bucket to be converted from couchstore to magma bucket.
      At the beginning all KV nodes for the bucket have couchstore as the storage engine underneath. Any new KV node that gets added to the cluster and rebalanced in will have the storage engine for that bucket set to magma, the older nodes will still have couchstore as the storage engine. One by one or all at once we could move the bucket to eventually have only magma storage engine underneath. This approach has the obvious disadvantage of having to perform multiple rebalances and move data across the network to achieve migration to magma, but the simplicity in development might outweigh the one time pain to migrate.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ankush.sharma Ankush Sharma
              Abhijeeth.Nuthan Abhijeeth Nuthan
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty