Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6382

Bucket deletion fails to delete on-disk data and indexes

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.0-beta
    • Fix Version/s: 2.0
    • Component/s: ns_server
    • Security Level: Public
    • Environment:
      Centos 64 bit
      1613

      Description

      Deleted default-p and default-s buckets, while the indexing/compaction was still in progress

      Post deletion of these buckets, we stil have dangling databases (eg from node 17 and node 20)
      On node 20 (default bucket is not deleted), on node 17 (default-s bucket is not deleted)
      the indexes for these databases still exists on disk.

      node 20
      ls -latr /data/
      total 76
      drwxr-xr-x. 3 couchbase couchbase 4096 Aug 19 17:43 .bucket-0
      rw-rr-. 1 couchbase couchbase 4175 Aug 20 16:56 _users.couch.1
      rw-rr-. 1 couchbase couchbase 4175 Aug 20 16:56 _replicator.couch.1
      drwxr-xr-x. 3 couchbase couchbase 4096 Aug 20 17:38 .default
      dr-xr-xr-x. 26 root root 4096 Aug 20 19:22 ..
      drwxr-xr-x. 2 couchbase couchbase 4096 Aug 21 15:03 .default-p
      drwxr-xr-x. 3 couchbase couchbase 4096 Aug 22 13:05 @indexes
      drwxr-xr-x. 2 couchbase couchbase 4096 Aug 22 13:05 .default-s
      drwxr-xr-x. 2 couchbase couchbase 4096 Aug 22 13:09 ._replicator_design
      drwxr-xr-x. 11 couchbase couchbase 4096 Aug 22 13:09 .
      drwxr-xr-x. 2 couchbase couchbase 12288 Aug 22 13:11 default
      drwxr-xr-x. 2 couchbase couchbase 4096 Aug 22 13:28 go
      drwxr-xr-x. 70 couchbase couchbase 4096 Aug 22 13:28 .delete

      node 17
      Dont see message like following for the default-s bucket here
      [ns_server:info,2012-08-22T13:12:25.807,ns_1@10.3.121.17:'ns_memcached-default':ns_storage_conf:delete_couch_database:410]Deleting database <<"default/master">>: ok
      [ns_server:info,2012-08-22T13:12:25.808,ns_1@10.3.121.17:'ns_memcached-default':ns_storage_conf:delete_databases_and_files:462]Couch dbs are deleted. Proceeding with bucket directory

      node 20
      Dont see message like following for the default bucket here
      [ns_server:info,2012-08-22T13:05:51.167,ns_1@10.3.121.20:'ns_memcached-default-s':ns_storage_conf:delete_databases_and_files:462]Couch dbs are deleted. Proceeding with bucket directory

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        karan Karan Kumar (Inactive) created issue -
        Show
        karan Karan Kumar (Inactive) added a comment - https://s3.amazonaws.com/bugdb/jira/deletion_bug/deletion_node17.zip https://s3.amazonaws.com/bugdb/jira/deletion_bug/deletion_node20.zip
        Hide
        karan Karan Kumar (Inactive) added a comment -

        Dont see this either in the logs

        Error ->
        ale:error(?USER_LOGGER, "Unable to delete some DBs for bucket ~s. Leaving bucket directory undeleted~n~p", [Bucket, Error]),
        Error

        Show
        karan Karan Kumar (Inactive) added a comment - Dont see this either in the logs Error -> ale:error(?USER_LOGGER, "Unable to delete some DBs for bucket ~s. Leaving bucket directory undeleted~n~p", [Bucket, Error] ), Error
        Hide
        karan Karan Kumar (Inactive) added a comment -

        To update, on all the nodes, out of 2 buckets, we have node deleting only 1 bucket.

        Show
        karan Karan Kumar (Inactive) added a comment - To update, on all the nodes, out of 2 buckets, we have node deleting only 1 bucket.
        farshid Farshid Ghods (Inactive) made changes -
        Field Original Value New Value
        Fix Version/s 2.0-beta [ 10113 ]
        Fix Version/s 2.0 [ 10114 ]
        farshid Farshid Ghods (Inactive) made changes -
        Labels 2.0-beta-release-notes
        farshid Farshid Ghods (Inactive) made changes -
        Fix Version/s 2.0 [ 10114 ]
        Fix Version/s 2.0-beta [ 10113 ]
        karan Karan Kumar (Inactive) made changes -
        Labels 2.0-beta-release-notes 2.0-beta-release-notes system-test
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Looks like this happened because of timeout listing vbuckets to delete, but we don't do it via couch_server calls anymore. Thus fixed by unrelated commit.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Looks like this happened because of timeout listing vbuckets to delete, but we don't do it via couch_server calls anymore. Thus fixed by unrelated commit.
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Cannot Reproduce [ 5 ]
        farshid Farshid Ghods (Inactive) made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            alkondratenko Aleksey Kondratenko (Inactive)
            Reporter:
            karan Karan Kumar (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes