Details
-
Bug
-
Resolution: Duplicate
-
Major
-
7.2.1
-
7.1.5-3877 to 7.2.1-5878 on AWS
-
Untriaged
-
0
-
Unknown
Description
It's a 7 node cluster ( 3 KV + 2 GSI + 2 Query) nodes with c5.4xlarge instance type.
The KV nodes are of 200 GB disk and GSI nodes of 400 GB disk.
The cluster was upgraded from 7.1.5-3877 to 7.2.1-5878. Post upgrade the RR seems to have dropped to a very low number (0 to 1 %). The memory_rss seems to be under quota.
"memory_rss": 24873295872
"memory_quota": 26843545600
Not really sure if it's a bug or if this is expected behavior, but just wanted to be sure there's nothing else wrong with the cluster. Post upgrade, one bucket was flushed and deleted entirely to help with the RR (not sure if this is the correct way to do this), but it didn't really help.
Cbcollect before upgrade ->
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_1/collectinfo-2023-07-24T083225-ns_1%40svc-d-node-001.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_1/collectinfo-2023-07-24T083225-ns_1%40svc-d-node-002.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_1/collectinfo-2023-07-24T083225-ns_1%40svc-d-node-003.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_1/collectinfo-2023-07-24T083225-ns_1%40svc-i-node-004.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_1/collectinfo-2023-07-24T083225-ns_1%40svc-i-node-005.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_1/collectinfo-2023-07-24T083225-ns_1%40svc-q-node-006.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_1/collectinfo-2023-07-24T083225-ns_1%40svc-q-node-007.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
cbcollect post upgrade ->
https://cb-engineering.s3.amazonaws.com/Systest_Jul24_2/collectinfo-2023-07-25T035826-ns_1%40svc-d-node-008.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/Systest_Jul24_2/collectinfo-2023-07-25T035826-ns_1%40svc-d-node-009.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/Systest_Jul24_2/collectinfo-2023-07-25T035826-ns_1%40svc-d-node-010.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/Systest_Jul24_2/collectinfo-2023-07-25T035826-ns_1%40svc-i-node-012.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/Systest_Jul24_2/collectinfo-2023-07-25T035826-ns_1%40svc-i-node-013.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/Systest_Jul24_2/collectinfo-2023-07-25T035826-ns_1%40svc-q-node-011.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/Systest_Jul24_2/collectinfo-2023-07-25T035826-ns_1%40svc-q-node-014.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
cbcollect latest ->
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_3/collectinfo-2023-07-25T102644-ns_1%40svc-d-node-008.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_3/collectinfo-2023-07-25T102644-ns_1%40svc-d-node-009.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_3/collectinfo-2023-07-25T102644-ns_1%40svc-d-node-010.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_3/collectinfo-2023-07-25T102644-ns_1%40svc-i-node-012.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_3/collectinfo-2023-07-25T102644-ns_1%40svc-i-node-013.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_3/collectinfo-2023-07-25T102644-ns_1%40svc-q-node-011.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
https://cb-engineering.s3.amazonaws.com/SysTest_24Jul_3/collectinfo-2023-07-25T102644-ns_1%40svc-q-node-014.x6xy5nt4xj5p-vn.sandbox.nonprod-project-avengers.com.zip |
Supportal link ->
https://supportal.couchbase.com/customer/systest_24jul_1 |
A caveat ->
The pre-upgrade cbcollect is a little old. I did not take a cbcollect just before the upgrade unfortunately. But according to this, the avg_resident ratio is around 11%.