Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.0.4
-
Untriaged
-
1
-
Unknown
Description
Steps:
1. Create a 1node 7.0.4 (7.0.4-7265) cluster with all services in it - kv,gsi,query,eventing,analytics,backup,fts
2. Install travel-sample bucket
3. Add a local filesystem backup repo to the node using backup service
4. Run a full backup
5. Add a 7.1.0 (7.1.0-2549) node to the cluster with all services except backup - kv,gsi,query,eventing,analytics,fts
Backup service crash is observed in the logs in 7.0.4 node:
Service 'backup' exited with status 1. Restarting. Messages:
|
2022-04-28T15:00:34.839-07:00 INFO (Main) Running node version backup-7.0.4-7265- with options: -http-port=8097 -grpc-port=9124 -https-port=18097 -cert-path=/opt/couchbase/var/lib/couchbase/config/memcached-cert.pem -key-path=/opt/couchbase/var/lib/couchbase/config/memcached-key.pem -ipv4=required -ipv6=optional -cbm=/opt/couchbase/bin/cbbackupmgr -node-uuid=6c014899d3d1c06ab66a2fc515258482 -public-address=10.112.210.101 -admin-port=8091 -log-file=none -log-level=debug -integrated-mode -integrated-mode-host=http://127.0.0.1:8091 -secure-integrated-mode-host=https://127.0.0.1:18091 -integrated-mode-user=@backup -default-collect-logs-path=/opt/couchbase/var/lib/couchbase/tmp -cbauth-host=127.0.0.1:8091
|
2022-04-28T15:00:34.839-07:00 INFO (Main) Initialized logger {"log level": "debug"}
|
2022-04-28T15:00:34.839-07:00 INFO (Main) Getting credentials
|
2022-04-28T15:00:34.839-07:00 ERROR (Main) Failed to run node {"err": "could not get credentials via cbauth: CBAuth database is stale: last reason: dial tcp 127.0.0.1:8091: connect: connection refused"}
|
2022/04/28 15:00:34 revrpc: Got error (dial tcp 127.0.0.1:8091: connect: connection refused) and will retry in 1s
|
This might be same as MB-51892
6. click on rebalance button - rebalance goes thru fine - we can see that the cluster is in mixed mode
7. Try to view the backup page from 7.1.0 node now - an error is thrown like this:
8. Failover the 7.0.4 node now - rebalance
9. Upgrade the 7.0.4 node to 7.1.0 and add it back with all services including backup - backup page will be fine now on both nodes and no issues
Attaching logs collected in mixed mode
