Analytics service down while upgrade

Test script steps :

1. Deploys a cluster - 3 node

1. Loads Buckets, scopes and 5 collections onto it

1. Loads docs onto it so the CPU usage goes up

1. Triggers a Scale Out to 5 Nodes

1. After a successful ScaleOut, Triggers an upgrade to the version mentioned

1. Post successful upgrade, triggers a Scale-In back to 3 nodes.

1. Destroys the cluster.

Observations :

• An AWS cluster was deployed with the following specifications :

• • ID : 1c8b0fa9-c71a-4552-a5dd-ef99d5c7db80

• • Deploy Image : couchbase-cloud-server-7.2.4-7070-x86_64-v1.0.27

• • Deploy version : 7.2.4

• • Upgrade to : 7.2.6

• • Upgrade image : couchbase-cloud-server-7.2.6-8101-x86_64-v1.0.34

• The cluster has been stuck in a Upgrading state since 17 hours

• The server logs show that the cluster has completed the upgrade successfully.

• Analytics service seems to be down as per the speculation from the AV :  https://couchbasecloud.atlassian.net/browse/AV-83269

Note :

Similar behaviour was also observed with the GCP cluster.

Specs :

• ID : b1879ce3-010f-4ef4-bf81-474bcf44f572

• Deployed Image : couchbase-cloud-server-7-2-4-7070-v1-0-27

• Deployed version : 7.2.4 (same as AWS above)

• Upgrade version : 7.2.6 (same as AWS above)

• Upgrade Image : couchbase-cloud-server-7-2-6-8101-v1-0-34