Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6222

rebalance failed because memcached died on windows

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 2.0-beta
    • 2.0-beta
    • couchbase-bucket
    • Security Level: Public
    • None
    • windows server r2 2008
      build #1583

    Description

      1 node, 1 bucket, 3 items, rebalance in 1 node

        • Last message in was {#Port<0.18432>,
          Unknown macro: {exit_status,255}

          }

        • When Server state ==
          Unknown macro: {state,#Port<0.18432>,memcached, {["114 3: Schedule cleanup of \"eq_tapq:anon_78\"", "114 3: TAP (Producer) eq_tapq:replication_building_590_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"initial_vbucket_stream\" and vbucket 590", "114 3: TAP (Producer) eq_tapq:replication_building_590_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "114 3: TAP (Producer) eq_tapq:replication_building_590_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "114 3: TAP (Producer) eq_tapq:replication_building_590_'ns_1@10.1.3.147' - Schedule the backfill for vbucket 590", "114 3: TAP (Producer) eq_tapq:replication_building_589_'ns_1@10.1.3.147' - Connection is closed by force.", "114 3: TAP (Producer) eq_tapq:rebalance_589 - Clear the tap queues by force", "114 3: Schedule cleanup of \"eq_tapq:rebalance_589\"", "114 3: TAP (Producer) eq_tapq:rebalance_589 - disconnected", "114 3: TAP takeover is completed. Disconnecting tap stream <eq_tapq:rebalance_589>", "114 3: TAP (Producer) eq_tapq:rebalance_589 - Sending TAP_VBUCKET_SET with vbucket 589 and state \"active\"", "114 3: TAP (Producer) eq_tapq:rebalance_589 - VBucket <589> is going dead to complete vbucket takeover.", "114 3: TAP (Producer) eq_tapq:rebalance_589 - Sending TAP_VBUCKET_SET with vbucket 589 and state \"pending\"", "114 3: TAP (Producer) eq_tapq:rebalance_589 - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "114 3: TAP (Producer) eq_tapq:rebalance_589 - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "114 3: TAP (Producer) eq_tapq:replication_building_589_'ns_1@10.1.3.147' - disconnected, keep alive for 300 seconds", "114 3: TAP (Producer) eq_tapq:replication_building_589_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"close_backfill\" and vbucket 589", "114 3: TAP (Producer) eq_tapq:replication_building_589_'ns_1@10.1.3.147' - Backfill is completed with VBuckets 589, ", "114 3: TAP (Producer) eq_tapq:rebalance_588 - Clear the tap queues by force", "114 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - Clear the tap queues by force", "114 3: Schedule cleanup of \"eq_tapq:rebalance_588\"", "114 3: Schedule cleanup of \"eq_tapq:anon_77\"", "114 3: TAP (Producer) eq_tapq:replication_building_589_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"initial_vbucket_stream\" and vbucket 589", "114 3: TAP (Producer) eq_tapq:replication_building_589_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "114 3: TAP (Producer) eq_tapq:replication_building_589_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "114 3: TAP (Producer) eq_tapq:replication_building_589_'ns_1@10.1.3.147' - Schedule the backfill for vbucket 589", "114 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - Connection is closed by force.", "114 3: TAP (Producer) eq_tapq:rebalance_588 - disconnected", "114 3: TAP takeover is completed. Disconnecting tap stream <eq_tapq:rebalance_588>", "114 3: TAP (Producer) eq_tapq:rebalance_588 - Sending TAP_VBUCKET_SET with vbucket 588 and state \"active\"", "114 3: TAP (Producer) eq_tapq:rebalance_588 - VBucket <588> is going dead to complete vbucket takeover.", "114 3: TAP (Producer) eq_tapq:rebalance_588 - Sending TAP_VBUCKET_SET with vbucket 588 and state \"pending\"", "114 3: TAP (Producer) eq_tapq:rebalance_588 - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "114 3: TAP (Producer) eq_tapq:rebalance_588 - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "114 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - disconnected, keep alive for 300 seconds", "113 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"close_backfill\" and vbucket 588", "113 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - Backfill is completed with VBuckets 588, ", "113 3: TAP (Producer) eq_tapq:rebalance_587 - Clear the tap queues by force"], ["112 3: TAP (Producer) eq_tapq:replication_building_585_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "112 3: TAP (Producer) eq_tapq:replication_building_585_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"initial_vbucket_stream\" and vbucket 585", "112 3: Schedule cleanup of \"eq_tapq:anon_73\"", "112 3: Schedule cleanup of \"eq_tapq:rebalance_584\"", "112 3: TAP (Producer) eq_tapq:replication_building_584_'ns_1@10.1.3.147' - Clear the tap queues by force", "112 3: TAP (Producer) eq_tapq:rebalance_584 - Clear the tap queues by force", "112 3: TAP (Producer) eq_tapq:replication_building_585_'ns_1@10.1.3.147' - Backfill is completed with VBuckets 585, ", "112 3: TAP (Producer) eq_tapq:replication_building_585_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"close_backfill\" and vbucket 585", "112 3: TAP (Producer) eq_tapq:replication_building_585_'ns_1@10.1.3.147' - disconnected, keep alive for 300 seconds", "113 3: TAP (Producer) eq_tapq:rebalance_585 - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:rebalance_585 - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:rebalance_585 - Sending TAP_VBUCKET_SET with vbucket 585 and state \"pending\"", "113 3: TAP (Producer) eq_tapq:rebalance_585 - VBucket <585> is going dead to complete vbucket takeover.", "113 3: TAP (Producer) eq_tapq:rebalance_585 - Sending TAP_VBUCKET_SET with vbucket 585 and state \"active\"", "113 3: TAP takeover is completed. Disconnecting tap stream <eq_tapq:rebalance_585>", "113 3: TAP (Producer) eq_tapq:rebalance_585 - disconnected", "113 3: Schedule cleanup of \"eq_tapq:rebalance_585\"", "113 3: TAP (Producer) eq_tapq:rebalance_585 - Clear the tap queues by force", "113 3: TAP (Producer) eq_tapq:replication_building_585_'ns_1@10.1.3.147' - Connection is closed by force.", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - Schedule the backfill for vbucket 586", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"initial_vbucket_stream\" and vbucket 586", "113 3: Schedule cleanup of \"eq_tapq:anon_74\"", "113 3: TAP (Producer) eq_tapq:replication_building_585_'ns_1@10.1.3.147' - Clear the tap queues by force", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - Backfill is completed with VBuckets 586, ", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"close_backfill\" and vbucket 586", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - disconnected, keep alive for 300 seconds", "113 3: TAP (Producer) eq_tapq:rebalance_586 - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:rebalance_586 - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:rebalance_586 - Sending TAP_VBUCKET_SET with vbucket 586 and state \"pending\"", "113 3: TAP (Producer) eq_tapq:rebalance_586 - VBucket <586> is going dead to complete vbucket takeover.", "113 3: TAP (Producer) eq_tapq:rebalance_586 - Sending TAP_VBUCKET_SET with vbucket 586 and state \"active\"", "113 3: TAP takeover is completed. Disconnecting tap stream <eq_tapq:rebalance_586>", "113 3: TAP (Producer) eq_tapq:rebalance_586 - disconnected", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - Connection is closed by force.", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - Schedule the backfill for vbucket 587", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"initial_vbucket_stream\" and vbucket 587", "113 3: Schedule cleanup of \"eq_tapq:anon_75\"", "113 3: Schedule cleanup of \"eq_tapq:rebalance_586\"", "113 3: TAP (Producer) eq_tapq:replication_building_586_'ns_1@10.1.3.147' - Clear the tap queues by force", "113 3: TAP (Producer) eq_tapq:rebalance_586 - Clear the tap queues by force", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - Backfill is completed with VBuckets 587, ", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"close_backfill\" and vbucket 587", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - disconnected, keep alive for 300 seconds", "113 3: TAP (Producer) eq_tapq:rebalance_587 - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:rebalance_587 - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:rebalance_587 - Sending TAP_VBUCKET_SET with vbucket 587 and state \"pending\"", "113 3: TAP (Producer) eq_tapq:rebalance_587 - VBucket <587> is going dead to complete vbucket takeover.", "113 3: TAP (Producer) eq_tapq:rebalance_587 - Sending TAP_VBUCKET_SET with vbucket 587 and state \"active\"", "113 3: TAP takeover is completed. Disconnecting tap stream <eq_tapq:rebalance_587>", "113 3: TAP (Producer) eq_tapq:rebalance_587 - disconnected", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - Connection is closed by force.", "113 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - Schedule the backfill for vbucket 588", "113 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"opaque_enable_auto_nack\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"enable_checkpoint_sync\" and vbucket 0", "113 3: TAP (Producer) eq_tapq:replication_building_588_'ns_1@10.1.3.147' - Sending TAP_OPAQUE with command \"initial_vbucket_stream\" and vbucket 588", "113 3: Schedule cleanup of \"eq_tapq:anon_76\"", "113 3: Schedule cleanup of \"eq_tapq:rebalance_587\"", "113 3: TAP (Producer) eq_tapq:replication_building_587_'ns_1@10.1.3.147' - Clear the tap queues by force"]}, undefined,[],0,true}
        • Reason for termination ==
        • {abnormal,255}

          [error_logger:error,2012-08-14T10:24:04.859,ns_1@10.1.3.146:error_logger:ale_error_logger_handler:log_report:72]
          =========================CRASH REPORT=========================
          crasher:
          initial call: ns_port_server:init/1
          pid: <0.22831.0>
          registered_name: ns_port_memcached
          exception exit: {abnormal,255}

          in function gen_server:terminate/6
          ancestors: [<0.22830.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
          <0.67.0>]
          messages: [

          {'EXIT',#Port<0.18432>,normal}

          ]
          links: [<0.22830.0>]
          dictionary: []
          trap_exit: true
          status: running
          heap_size: 121393
          stack_size: 24
          reductions: 109012
          neighbours:

      Attachments

        1. 10.1.3.146-8091-diag.txt.gz
          1009 kB
        2. 10.1.3.147-8091-diag.txt.gz
          820 kB
        3. error.1
          302 kB
        4. info.1
          7.99 MB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            jin Jin Lim (Inactive)
            iryna iryna
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty