Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
1.8.1
-
Security Level: Public
-
None
-
windows 2008 r2 64bit
Description
Install couchbase server 1.8.1r-788 on 6 nodes windows server 2008 R2 64bit.
The run the test rebalancetests.RebalanceInOutWithParallelLoad.test_load
The test failed due to rebalance failed (memcached crashed)
Check log page, see these errors:
Rebalance exited with reason {{change_filter_failed,
{'EXIT',
{{badmatch,failed},
[
,
{misc,'-executing_on_new_process/1-fun-0-', 3}]}}},
[
,
{ns_vbm_sup,'-set_replicas/3-fun-2-',5},
{lists,foldl,3},
{ns_vbm_sup,set_replicas,3},
{ns_vbm_sup,'-set_replicas/2-fun-1-',3},
{lists,foreach,2},
{ns_vbm_sup,apply_changes,2},
{ns_vbucket_mover,sync_replicas,0}]}
Control connection to memcached on 'ns_1@10.1.3.82' disconnected: {{badmatch,
{error,
closed}},
[
,
{mc_client_binary, stats,4},
{ns_memcached, do_handle_call, 3},
{ns_memcached, handle_call, 3},
{gen_server, handle_msg, 5},
{proc_lib, init_p_do_apply, 3}]}
Port server memcached on node 'ns_1@10.1.3.82' exited with status 38. Restarting. Messages: TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_31_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 31,
TAP (Producer) eq_tapq:replication_building_31_'ns_1@10.1.3.85' - Disconnecting pending connection
Failed to lookup TAP connection.. Disconnecting
TAP (Producer) eq_tapq:replication_building_31_'ns_1@10.1.3.84' - Disconnecting pending connection
Failed to lookup TAP connection.. Disconnecting
TAP (Producer) eq_tapq:replication_building_31_'ns_1@10.1.3.87' - Disconnecting pending connection
Failed to lookup TAP connection.. Disconnecting
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.87 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.87 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_34_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 34,
TAP (Producer) eq_tapq:replication_building_34_'ns_1@10.1.3.87' - Disconnecting pending connection
Failed to lookup TAP connection.. Disconnecting
TAP (Producer) eq_tapq:replication_building_34_'ns_1@10.1.3.85' - Disconnecting pending connection
Failed to lookup TAP connection.. Disconnecting
TAP (Producer) eq_tapq:replication_building_34_'ns_1@10.1.3.84' - Disconnecting pending connection
Failed to lookup TAP connection.. Disconnecting
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.87 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_35_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 35,
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.87 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_35_'ns_1@10.1.3.87' - Disconnecting pending connection
Unable to process tap ack. No producer found
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_36_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 36,
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_37_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 37,
TAP (Producer) eq_tapq:replication_building_37_'ns_1@10.1.3.84' - Disconnecting pending connection
Failed to lookup TAP connection.. Disconnecting
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_38_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 38,
TAP (Producer) eq_tapq:replication_building_38_'ns_1@10.1.3.84' - Disconnecting pending connection
Unable to process tap ack. No producer found
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_54_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 54,
TAP (Producer) eq_tapq:replication_building_54_'ns_1@10.1.3.87' - Disconnecting pending connection
Failed to lookup TAP connection.. Disconnecting
TAP (Producer) eq_tapq:replication_building_54_'ns_1@10.1.3.85' - Disconnecting pending connection
Unable to process tap ack. No producer found
TAP (Producer) eq_tapq:replication_building_54_'ns_1@10.1.3.84' - Disconnecting pending connection
Unable to process tap ack. No producer found
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.87 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_55_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 55,
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.87 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.85 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_building_56_'ns_1@10.1.3.87' - Backfill is completed with VBuckets 56,
TAP (Producer) eq_tapq:replication_building_56_'ns_1@10.1.3.85' - Disconnecting pending connection
Unable to process tap ack. No producer found
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.87 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.87 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.84 - Connection is re-established. Rollback unacked messages...
TAP (Producer) eq_tapq:replication_ns_1@10.1.3.86 - Explicit ack of nonexisting entry (#436253)