Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.1.2
-
7.1.2-3432-enterprise
-
Untriaged
-
Centos 64-bit
-
-
1
-
Unknown
Description
Build: 7.1.2-3432
Scenario:
1. Enable ipv6-only on s22401
2022-09-18 05:32:43 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] running command.raw on s22401-ip6.qe.couchbase.com: /opt/couchbase/bin/couchbase-cli ip-family -c s22401-ip6.qe.couchbase.com:8091 -u Administrator -p password --set --ipv6only
|
2022-09-18 05:32:44 | INFO | MainProcess | test_thread | [remote_util.execute_command_raw] command executed successfully with root
|
2022-09-18 05:32:44 | INFO | MainProcess | test_thread | [remote_util.log_command_output] Switched IP family for node: http://s22401-ip6.qe.couchbase.com:8091
|
2022-09-18 05:32:44 | INFO | MainProcess | test_thread | [remote_util.log_command_output] SUCCESS: Switched IP family of the cluster
|
2. Add second node (s22404) and rebalance the cluster.
Observing the rebalance fails with backup service crashing with status code 1.
{service_rebalance_failed,backup,
|
{agent_died,<0.10116.0>,
|
{linked_process_died,<0.10117.0>,
|
{'ns_1@s22401-ip6.qe.couchbase.com', {no_connection,\"backup-service_api\"}}}}}."}
|
From debug.log
=========================WARNING REPORT=========================
|
Description: "Authenticity is not established by certificate path validation"
|
Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"[ns_server:debug,2022-09-20T02:28:45.690-07:00,ns_1@s22401-ip6.qe.couchbase.com:json_rpc_connection-backup-cbauth<0.4988.0>:json_rpc_connection:init:68]Observed revrpc connection: label "backup-cbauth", handling process <0.4988.0>
|
[ns_server:debug,2022-09-20T02:28:45.690-07:00,ns_1@s22401-ip6.qe.couchbase.com:menelaus_cbauth<0.552.0>:menelaus_cbauth:handle_cast:101]Observed json rpc process {"backup-cbauth",<0.4988.0>} started
|
[ns_server:debug,2022-09-20T02:28:45.728-07:00,ns_1@s22401-ip6.qe.couchbase.com:json_rpc_connection-backup-cbauth<0.4988.0>:json_rpc_connection:handle_info:124]Socket closed
|
[ns_server:debug,2022-09-20T02:28:45.729-07:00,ns_1@s22401-ip6.qe.couchbase.com:menelaus_cbauth<0.552.0>:menelaus_cbauth:handle_info:136]Observed json rpc process {"backup-cbauth",<0.4988.0>} died with reason shutdown
|
[user:info,2022-09-20T02:28:45.729-07:00,ns_1@s22401-ip6.qe.couchbase.com:<0.404.0>:ns_log:consume_log:76]Service 'backup' exited with status 1. Restarting. Messages:
|
2022-09-20T02:28:45.722-07:00 INFO (Main) Initialized logger {"log level": "debug"}
|
2022-09-20T02:28:45.722-07:00 INFO (Main) Getting credentials
|
2022-09-20T02:28:45.722-07:00 DEBUG (Main) File limit info {"curr": 200000, "max": 200000, "err": null}
|
2022-09-20T02:28:45.722-07:00 DEBUG (REST) (Attempt 1) (GET) Dispatching request to 'http://127.0.0.1:8091/pools/default/nodeServices'
|
2022-09-20T02:28:45.724-07:00 ERROR (REST) (Attempt 1) (GET) Failed to perform request to 'http://127.0.0.1:8091/pools/default/nodeServices': Get "http://127.0.0.1:8091/pools/default/nodeServices": dial tcp 127.0.0.1:8091: connect: connection refused
|
2022-09-20T02:28:45.725-07:00 WARN (REST) failed to bootstrap client, will retry: failed to execute request: Get "http://127.0.0.1:8091/pools/default/nodeServices": dial tcp 127.0.0.1:8091: connect: connection refused
|
2022-09-20T02:28:45.725-07:00 ERROR (Main) Failed to run node {"err": "could not create REST client: failed to bootstrap client: failed to connect to any host(s) from the connection string, check username and password"}[ns_server:debug,2022-09-20T02:28:45.875-07:00,ns_1@s22401-ip6.qe.couchbase.com:cb_dist<0.211.0>:cb_dist:info_msg:872]cb_dist: Accepted new connection from <0.216.0> DistCtrl #Port<0.468>: {con,
|
#Ref<0.1814526785.2013528065.208661>,
|
inet6_tcp_dist,
|
undefined,
|
undefined}
|
[
|