Details
-
Bug
-
Resolution: Duplicate
-
Critical
-
Cheshire-Cat
-
6.6.2->9588 to 7.0.0-5033
-
Untriaged
-
1
-
No
Description
Steps to repro
1. Run longevity system test on 6.6.2 for 3 days. It initially had 27 node cluster.
./sequoia -client 172.23.96.162:2375 -provider file:centos_third_cluster.yml -test tests/integration/test_allFeatures_madhatter_durability.yml -scope tests/integration/scope_Xattrs_Madhatter.yml -scale 3 -repeat 0 -log_level 0 -version 6.6.2-9588 -skip_setup=false -skip_test=false -skip_teardown=true -skip_cleanup=false -continue=false -collect_on_error=false -stop_on_error=false -duration=604800 -show_topology=true
|
2. Swap rebalance 6 nodes one of each service type.
3. Now failover 6 nodes, one of each service type.
4. sysemctl stop couchbas-server on 6 nodes.
5. Install 7.0.0 on all of them.
6. Do a recovery + rebalance
Then I saw multiple failures on cbas nodes as shown below.
ns_1@172.23.105.90 1:07:22 AM 26 Apr, 2021
Service 'cbas' exited with status 1. Restarting. Messages:
|
2021-04-26T01:07:20.410-07:00 CRIT CBAS.cbas my node is in topology but not registered in cbauth db
|
2021-04-26T01:07:20.410-07:00 WARN CBAS.cbas driver bootstrap attempt failed: my node is in topology but not registered in cbauth db
|
2021-04-26T01:07:20.410-07:00 INFO CBAS.cbas driver bootstrap failed; will retry in 1s (2 retries remaining)
|
2021-04-26T01:07:21.410-07:00 CRIT CBAS.cbas my node is in topology but not registered in cbauth db
|
2021-04-26T01:07:21.410-07:00 WARN CBAS.cbas driver bootstrap attempt failed: my node is in topology but not registered in cbauth db
|
2021-04-26T01:07:21.410-07:00 INFO CBAS.cbas driver bootstrap failed; will retry in 1s (1 retries remaining)
|
2021-04-26T01:07:22.411-07:00 CRIT CBAS.cbas my node is in topology but not registered in cbauth db
|
2021-04-26T01:07:22.411-07:00 WARN CBAS.cbas driver bootstrap attempt failed: my node is in topology but not registered in cbauth db
|
2021-04-26T01:07:22.411-07:00 FATA CBAS.cbas unable to complete driver bootstrap after 60 attempt(s) (last failure: my node is in topology but not registered in cbauth db)
|
cbcollect_info attached.
Attachments
Issue Links
- duplicates
-
-
- Closed
-