Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
7.6.0
-
7.6.0-2090
-
Untriaged
-
0
-
Unknown
Description
observing some instance of internal server error when adding a node -
Debian case:
Adding a node to 172.23.122.140
https://cb-engineering.s3.amazonaws.com/test/collectinfo-2024-02-09T153847-ns_1%40172.23.122.140.zip
https://cb-engineering.s3.amazonaws.com/7.6.0/test/collectinfo-2024-02-09T112913-ns_1@cb.local.zip
[ns_server:info,2024-02-09T03:12:47.929-08:00,ns_1@172.23.122.140:leader_registry<0.1749.0>:leader_registry:handle_down:286]Process <0.1780.0> registered as 'tombstone_purger' terminated. [ns_server:info,2024-02-09T03:12:47.929-08:00,ns_1@172.23.122.140:leader_registry<0.1749.0>:leader_registry:handle_down:286]Process <0.1775.0> registered as 'auto_failover' terminated. [ns_server:error,2024-02-09T03:12:47.929-08:00,ns_1@172.23.122.140:<0.3426.0>:menelaus_util:reply_server_error_before_close:223]Server error during processing: ["web request failed", {path,"/controller/addNode"}, {method,'POST'}, {type,exit}, {what, {{{{{badmatch,{error,enoent}}, [{ns_ssl_services_setup,save_certs,6, [{file, "src/ns_ssl_services_setup.erl"}, {line,958}]}, {ns_ssl_services_setup, maybe_generate_certs,2, [{file, "src/ns_ssl_services_setup.erl"}, {line,908}]}, {ns_ssl_services_setup,handle_info,2, [{file, "src/ns_ssl_services_setup.erl"}, {line,757}]}, {gen_server,try_dispatch,4, [{file,"gen_server.erl"},{line,1123}]}, {gen_server,handle_msg,6, [{file,"gen_server.erl"},{line,1200}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,240}]}]}, {gen_server,call, [ns_ssl_services_setup,ping,300000]}}, {gen_server,call, [dist_manager, {adjust_my_address,"172.23.122.140", false,#Fun<ns_cluster.7.128832967>, #Fun<ns_cluster.12.128832967>}, infinity]}}, {gen_server,call, [ns_cluster, {add_node_to_group,https, "172.23.122.144",18091, #Fun<menelaus_web_cluster.29.106616858>,
|
this node will be available 172.23.122.140 for live debugging
Centos case
Adding a node to 172.23.136.227
https://cb-engineering.s3.amazonaws.com/test/collectinfo-2024-02-09T111544-ns_1%40172.23.136.227.zip
https://cb-engineering.s3.amazonaws.com/7.6.0/test/cbcollect.zip
[ns_server:info,2024-02-09T03:10:17.871-08:00,ns_1@172.23.136.227:ns_node_disco_events<0.567.0>:ns_node_disco_log:handle_event:40]ns_node_disco_log: nodes changed: ['ns_1@172.23.136.227'] [ns_server:warn,2024-02-09T03:10:17.872-08:00,ns_1@172.23.136.227:<0.6787.0>:leader_lease_acquire_worker:handle_lease_already_acquired:226]Failed to acquire lease from 'ns_1@172.23.136.227' because its already taken by {'ns_1@cb.local', <<"e7e6ec52929ab238c0f1c300ad7741c4">>} (valid for 13212ms) [ns_server:info,2024-02-09T03:10:17.875-08:00,ns_1@172.23.136.227:mb_master<0.807.0>:mb_master:terminate:247]Synchronously shutting down child mb_master_sup [ns_server:error,2024-02-09T03:10:17.875-08:00,ns_1@172.23.136.227:<0.6613.0>:menelaus_util:reply_server_error_before_close:223]Server error during processing: ["web request failed", {path,"/controller/addNode"}, {method,'POST'}, {type,exit}, {what, {{{{{badmatch,{error,enoent}}, [{ns_ssl_services_setup,save_certs,6, [{file, "src/ns_ssl_services_setup.erl"}, {line,958}]}, {ns_ssl_services_setup, maybe_generate_certs,2, [{file, "src/ns_ssl_services_setup.erl"}, {line,908}]}, {ns_ssl_services_setup,handle_info,2, [{file, "src/ns_ssl_services_setup.erl"}, {line,757}]}, {gen_server,try_dispatch,4, [{file,"gen_server.erl"},{line,1123}]}, {gen_server,handle_msg,6, [{file,"gen_server.erl"},{line,1200}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,240}]}]}, {gen_server,call, [ns_ssl_services_setup,ping,300000]}}, {gen_server,call, [dist_manager, {adjust_my_address,"172.23.136.227", false,#Fun<ns_cluster.7.128832967>, #Fun<ns_cluster.12.128832967>}, infinity]}}, {gen_server,call, [ns_cluster, {add_node_to_group,https, "172.23.136.230",18091, #Fun<menelaus_web_cluster.29.106616858>, undefined, [kv]}, 240000]}}}, {trace, [{gen_server,call,3, [{file,"gen_server.erl"},{line,385}]}, {ns_cluster,add_node_to_group,6, [{file,"src/ns_cluster.erl"},{line,83}]}, {menelaus_web_cluster, '-do_handle_add_node/2-fun-2-',8, [{file,"src/menelaus_web_cluster.erl"}, {line,878}]},
|
|