Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7569

[windows]unable to delete bucket , rebalance failed and master became unavailable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.1
    • Fix Version/s: 2.0.1
    • Component/s: ns_server
    • Security Level: Public
    • Environment:

      Description

      -t view.createdeleteview.CreateDeleteViewTests.incremental_rebalance_out_with_ddoc_ops,ddoc_ops=create,test_with_view=True,num_ddocs=2,num_views_per_ddoc=3,items=200000

      1 bucket/2 ddocs/3 views per ddoc/ 200000 items
      delete bucket and rebalance out 3 nodes

      2013-01-20 23:13:15.720 menelaus_web:102:warning:client-side error report(ns_1@10.3.3.38) - Client-side error-report for user "Administrator" on node 'ns_1@10.3.3.38':
      User-Agent:Python-httplib2/$Rev: 259 $
      Starting rebalance from test, ejected nodes [u'ns_1@10.3.2.243', u'ns_1@10.3.3.39']

      2013-01-20 23:13:15.767 ns_storage_conf:0:info:message(ns_1@10.3.3.38) - Deleting old data files of bucket "default"
      2013-01-20 23:13:17.782 menelaus_web:102:warning:client-side error report(ns_1@10.3.3.38) - Client-side error-report for user "Administrator" on node 'ns_1@10.3.3.38':
      User-Agent:Python-httplib2/$Rev: 259 $
      2013-01-20 23:14:22.746282 : incremental_rebalance_out_with_ddoc_ops finished

      2013-01-20 23:13:20.173 menelaus_sup:1:info:web start ok(ns_1@10.3.3.38) - Couchbase Server has started on web port 8091 on node 'ns_1@10.3.3.38'.
      2013-01-20 23:14:18.369 ns_orchestrator:4:info:message(ns_1@10.3.2.243) - Starting rebalance, KeepNodes = ['ns_1@10.3.3.38'], EjectNodes = ['ns_1@10.3.2.243',
      'ns_1@10.3.3.39']

      2013-01-20 23:14:18.400 ns_rebalancer:0:critical:message(ns_1@10.3.2.243) - Failed to cleanup old buckets on some nodes:
      [{'ns_1@10.3.3.38',
      {'EXIT',
      {badarg,
      [

      {ets,lookup,[couch_dbs_by_name,<<"default/">>]}

      ,

      {couch_server,all_known_databases_with_prefix,1}

      ,

      {ns_storage_conf,delete_databases_and_files,1}

      ,

      {ns_storage_conf,delete_disk_buckets_databases_loop,2}

      ,

      {rpc,'-handle_call_call/6-fun-0-',5}

      ]}}}]
      2013-01-20 23:14:18.400 ns_orchestrator:2:info:message(ns_1@10.3.2.243) - Rebalance exited with reason

      {buckets_cleanup_failed,['ns_1@10.3.3.38']}

      attaching logs

      1. Logfile.CSV
        18.22 MB
        Sriram Melkote
      # Subject Project Status CR V
      For Gerrit Dashboard: &For+MB-7569=message:MB-7569

        Activity

        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-couchdb-preview #560 (See http://qa.hq.northscale.net/job/github-couchdb-preview/560/)
        MB-7569 Ignore eacces errors on delete dir init (Revision dc873ddd49d011dd927a847212aa25c71f5ad69b)

        Result = SUCCESS
        Filipe David Borba Manana :
        Files :

        • src/couchdb/couch_file.erl
        • src/couchdb/couch_view.erl
        • src/couch_set_view/src/couch_set_view.erl
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-couchdb-preview #560 (See http://qa.hq.northscale.net/job/github-couchdb-preview/560/ ) MB-7569 Ignore eacces errors on delete dir init (Revision dc873ddd49d011dd927a847212aa25c71f5ad69b) Result = SUCCESS Filipe David Borba Manana : Files : src/couchdb/couch_file.erl src/couchdb/couch_view.erl src/couch_set_view/src/couch_set_view.erl
        Hide
        FilipeManana Filipe Manana (Inactive) added a comment -

        Can you try again Iryna?
        thanks

        Show
        FilipeManana Filipe Manana (Inactive) added a comment - Can you try again Iryna? thanks
        Hide
        iryna iryna added a comment -

        tested build 2.0.1-150-rel
        fixed

        Show
        iryna iryna added a comment - tested build 2.0.1-150-rel fixed
        Hide
        iryna iryna added a comment -

        build 2.0.1-150-rel doesn't have this issue

        Show
        iryna iryna added a comment - build 2.0.1-150-rel doesn't have this issue
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-couchdb-preview #569 (See http://qa.hq.northscale.net/job/github-couchdb-preview/569/)
        Revert "MB-7569 Ignore eacces errors on delete dir init" (Revision 24d1a7a3b29aa10aea76f2536d52a6c5f047e9d1)

        Result = SUCCESS
        Filipe David Borba Manana :
        Files :

        • src/couch_set_view/src/couch_set_view.erl
        • src/couchdb/couch_file.erl
        • src/couchdb/couch_view.erl
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-couchdb-preview #569 (See http://qa.hq.northscale.net/job/github-couchdb-preview/569/ ) Revert " MB-7569 Ignore eacces errors on delete dir init" (Revision 24d1a7a3b29aa10aea76f2536d52a6c5f047e9d1) Result = SUCCESS Filipe David Borba Manana : Files : src/couch_set_view/src/couch_set_view.erl src/couchdb/couch_file.erl src/couchdb/couch_view.erl

          People

          • Assignee:
            iryna iryna
            Reporter:
            iryna iryna
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes