Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4564

Don't delete database files when node leaves cluster.

    Details

      Description

      A recent bug brought up a concern around the process of deleting the on-disk data files when a node is failed over or removed from the cluster.

      I think it would be a better approach to preserve this data in the event that it is needed. Perhaps moving them out of the way and messaging to the user that they can clean them up manually when needed.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        perry Perry Krug created issue -
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Perry, you want files preserved for eternity or we can/should get rid of them when node is re-initialized or joins some cluster ?

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Perry, you want files preserved for eternity or we can/should get rid of them when node is re-initialized or joins some cluster ?
        Hide
        perry Perry Krug added a comment -

        I think the join cluster could be a fine situation to clean them up. What do you mean by re-initialized? This is really only needed as a fail-safe situation in case we need to recover the data from the node that was failed over or removed.

        Show
        perry Perry Krug added a comment - I think the join cluster could be a fine situation to clean them up. What do you mean by re-initialized? This is really only needed as a fail-safe situation in case we need to recover the data from the node that was failed over or removed.
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        that's when/if node is initialized via setup wizard as standalone cluster

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - that's when/if node is initialized via setup wizard as standalone cluster
        Hide
        perry Perry Krug added a comment -

        Oh, yes...then that's a good time as well too. My only concern would be that the current code path re-initializes the node when it gets failed over or removed so we'd have to change that behavior.

        Show
        perry Perry Krug added a comment - Oh, yes...then that's a good time as well too. My only concern would be that the current code path re-initializes the node when it gets failed over or removed so we'd have to change that behavior.
        steve Steve Yen made changes -
        Field Original Value New Value
        Fix Version/s 1.8.1 [ 10249 ]
        dipti Dipti Borkar made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ]
        dipti Dipti Borkar made changes -
        Fix Version/s 1.8.1 [ 10295 ]
        Fix Version/s 1.8.2 [ 10249 ]
        Hide
        steve Steve Yen added a comment -

        Aliaksey still needs final requirements on how this needs to behave.

        Show
        steve Steve Yen added a comment - Aliaksey still needs final requirements on how this needs to behave.
        Hide
        steve Steve Yen added a comment -

        Please document decision here.

        Show
        steve Steve Yen added a comment - Please document decision here.
        steve Steve Yen made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ] Dipti Borkar [ dipti ]
        steve Steve Yen made changes -
        Priority Major [ 3 ] Blocker [ 1 ]
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Aliaksey took this.

        Decision was to don't touch files when node is ejected/failed over. But rid of them either in the middle of wizard or when rebalance back in starts.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Aliaksey took this. Decision was to don't touch files when node is ejected/failed over. But rid of them either in the middle of wizard or when rebalance back in starts.
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Assignee Dipti Borkar [ dipti ] Aliaksey Artamonau [ aliaksey artamonau ]
        Hide
        perry Perry Krug added a comment -

        Could we also add a task to notify the user when the files are about to be deleted (either in the wizard or when adding back / rebalancing)?

        Show
        perry Perry Krug added a comment - Could we also add a task to notify the user when the files are about to be deleted (either in the wizard or when adding back / rebalancing)?
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        how you want user to be notifed? We can use user-visible log message or we can have alert-like thing.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - how you want user to be notifed? We can use user-visible log message or we can have alert-like thing.
        Hide
        perry Perry Krug added a comment -

        I'd specifically like something that makes the user confirm that they understand that proceeding with whatever action will result in the deleting of files they may want to save. A log message after the fact doesn't give them the ability to recover that data.

        Show
        perry Perry Krug added a comment - I'd specifically like something that makes the user confirm that they understand that proceeding with whatever action will result in the deleting of files they may want to save. A log message after the fact doesn't give them the ability to recover that data.
        Hide
        steve Steve Yen added a comment -

        alk predicts this evening ETA

        Show
        steve Steve Yen added a comment - alk predicts this evening ETA
        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - http://review.couchbase.org/14628
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-ns-server-2-0 #333 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/333/)
        MB-4564 Don't delete database files when node leaves cluster. (Revision ac961c9839396f64ef6c0ef56e5f298095a0da63)

        Result = SUCCESS
        Aliaksey Kandratsenka :
        Files :

        • src/ns_memcached.erl
        • src/ns_cluster.erl
        • src/menelaus_web_buckets.erl
        • src/menelaus_web.erl
        • src/ns_storage_conf.erl
        • src/ns_rebalancer.erl
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #333 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/333/ ) MB-4564 Don't delete database files when node leaves cluster. (Revision ac961c9839396f64ef6c0ef56e5f298095a0da63) Result = SUCCESS Aliaksey Kandratsenka : Files : src/ns_memcached.erl src/ns_cluster.erl src/menelaus_web_buckets.erl src/menelaus_web.erl src/ns_storage_conf.erl src/ns_rebalancer.erl
        farshid Farshid Ghods (Inactive) made changes -
        Summary Don't delete data files after failover or remove Don't delete database files when node leaves cluster.
        farshid Farshid Ghods (Inactive) made changes -
        Labels customer 1.8.1-release-notes customer

          People

          • Assignee:
            Aliaksey Artamonau Aliaksey Artamonau
            Reporter:
            perry Perry Krug
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes