Details
-
Bug
-
Resolution: User Error
-
Major
-
6.5.1
-
Untriaged
-
-
Yes
Description
Build : 6.5.1-6271
Test : -test tests/integration/test_allFeatures_madhatter_durability.yml -scope tests/integration/scope_Xattrs_Madhatter.yml
Scale : 3
Day : 2 days
After bucket compaction got done at 2020-03-26T03:09:51, all buckets in the cluster got deleted starting from 2020-03-26T03:10:04. The test does not delete the buckets. Exact reason for bucket deletion could not be determined.
Following can be seen in the debug logs for 172.23.108.103
[ns_server:debug,2020-03-26T03:10:05.126-07:00,ns_1@172.23.108.103:<0.32096.0>:ns_pubsub:do_subscribe_link_continue:152]Parent process of subscription {ns_stats_event,<0.32095.0>} exited with reason shutdown
|
[ns_server:debug,2020-03-26T03:10:05.127-07:00,ns_1@172.23.108.103:<0.32094.0>:ns_pubsub:do_subscribe_link_continue:152]Parent process of subscription {ns_tick_event,<0.32093.0>} exited with reason shutdown
|
[ns_server:debug,2020-03-26T03:10:05.260-07:00,ns_1@172.23.108.103:terse_bucket_info_uploader-CUSTOMER<0.32077.0>:terse_bucket_info_uploader:refresh_cluster_config:93]Bucket CUSTOMER is dead
|
[ns_server:error,2020-03-26T03:10:05.280-07:00,ns_1@172.23.108.103:<0.12631.595>:dcp_proxy:handle_info:117]Socket #Port<0.634174> was closed. Closing myself. State = {state,
|
#Port<0.634174>,
|
{producer,
|
"replication:ns_1@172.23.104.61->ns_1@172.23.108.103:CUSTOMER",
|
'ns_1@172.23.104.61',
|
"CUSTOMER"},
|
undefined,<<>>,
|
dcp_producer_conn,
|
[],
|
#Port<0.634173>,
|
<0.20598.596>,
|
false}
|
[ns_server:debug,2020-03-26T03:10:05.281-07:00,ns_1@172.23.108.103:dcp_traffic_monitor<0.22105.728>:dcp_traffic_monitor:handle_info:82]Deleting Node:'ns_1@172.23.104.61' Bucket:"CUSTOMER" Pid:<0.12631.595>
|
[error_logger:error,2020-03-26T03:10:05.281-07:00,ns_1@172.23.108.103:error_logger<0.32.0>:ale_error_logger_handler:do_log:203]** Generic server <0.12631.595> terminating
|
** Last message in was {tcp_closed,#Port<0.634174>}
|
** When Server state == {state,#Port<0.634174>,
|
{producer,
|
"replication:ns_1@172.23.104.61->ns_1@172.23.108.103:CUSTOMER",
|
'ns_1@172.23.104.61',"CUSTOMER"},
|
undefined,<<>>,dcp_producer_conn,[],
|
#Port<0.634173>,<0.20598.596>,false}
|
** Reason for termination ==
|
** socket_closed
|
|
|
[error_logger:error,2020-03-26T03:10:05.282-07:00,ns_1@172.23.108.103:error_logger<0.32.0>:ale_error_logger_handler:do_log:203]
|
=========================CRASH REPORT=========================
|
crasher:
|
initial call: dcp_proxy:init/1
|
pid: <0.12631.595>
|
registered_name: []
|
exception exit: socket_closed
|
in function gen_server:handle_common_reply/8 (gen_server.erl, line 726)
|
ancestors: ['dcp_replicator-CUSTOMER-ns_1@172.23.104.61',
|
'dcp_sup-CUSTOMER','single_bucket_kv_sup-CUSTOMER',
|
ns_bucket_sup,ns_bucket_worker_sup,ns_server_sup,
|
ns_server_nodes_sup,<0.205.0>,ns_server_cluster_sup,
|
root_sup,<0.117.0>]
|
message_queue_len: 0
|
messages: []
|
links: [<0.19323.596>]
|
dictionary: []
|
trap_exit: false
|
status: running
|
heap_size: 1598
|
stack_size: 27
|
reductions: 36617911
|
neighbours:
|
|
|
[ns_server:debug,2020-03-26T03:10:05.282-07:00,ns_1@172.23.108.103:dcp_traffic_monitor<0.22105.728>:dcp_traffic_monitor:handle_info:82]Deleting Node:'ns_1@172.23.108.103' Bucket:"CUSTOMER" Pid:<0.20598.596>
|
[error_logger:error,2020-03-26T03:10:05.282-07:00,ns_1@172.23.108.103:error_logger<0.32.0>:ale_error_logger_handler:do_log:203]** Generic server 'dcp_replicator-CUSTOMER-ns_1@172.23.104.61' terminating
|
** Last message in was {'EXIT',<0.12631.595>,socket_closed}
|
** When Server state == {state,[{<0.20598.596>,#Port<0.634173>},
|
{<0.12631.595>,#Port<0.634174>}],
|
<0.20598.596>,
|
"replication:ns_1@172.23.104.61->ns_1@172.23.108.103:CUSTOMER",
|
'ns_1@172.23.104.61',"CUSTOMER"}
|
** Reason for termination ==
|
** socket_closed
|
|
|
[error_logger:error,2020-03-26T03:10:05.283-07:00,ns_1@172.23.108.103:error_logger<0.32.0>:ale_error_logger_handler:do_log:203]
|
=========================CRASH REPORT=========================
|
crasher:
|
initial call: dcp_replicator:init/1
|
pid: <0.19323.596>
|
registered_name: 'dcp_replicator-CUSTOMER-ns_1@172.23.104.61'
|
exception exit: socket_closed
|
in function gen_server:handle_common_reply/8 (gen_server.erl, line 726)
|
ancestors: ['dcp_sup-CUSTOMER','single_bucket_kv_sup-CUSTOMER',
|
ns_bucket_sup,ns_bucket_worker_sup,ns_server_sup,
|
ns_server_nodes_sup,<0.205.0>,ns_server_cluster_sup,
|
root_sup,<0.117.0>]
|
message_queue_len: 1
|
messages: [{'EXIT',<0.20598.596>,killed}]
|
links: [<0.32079.0>]
|
dictionary: []
|
trap_exit: true
|
status: running
|
heap_size: 2586
|
stack_size: 27
|
reductions: 1201008
|
|
The test and environment have been unchanged since 6.5.0, so this could be a regression.