Details
-
Bug
-
Resolution: Won't Fix
-
Major
-
None
-
None
-
None
-
28: Upgrades, small fixes, 30: Maintenance, CMOS, ARM
-
3
Description
If the root CA expires for a cluster that is already deployed then the Operator never rotates in the new Certs as it appears we've locked ourselves out.
The logs here show that Operator is attempting to Reload the certificate chain. Followed by a failure to do so because we are using 'https' on a connection with expired certs:
{"level":"info","ts":1659402872.166496,"logger":"cluster","msg":"Reloading certificate chain","cluster":"test-cccnh/test-couchbase-g5 |
2hv","name":"test-couchbase-g52hv-0000"} |
|
|
|
|
{"level":"debug","ts":1659402872.1791792,"logger":"api","msg":"http","cluster":"test-cccnh/test-couchbase-g52hv","method":"POST","url":"https://test-couchbase-g52hv-0000.test-couchbase-g52hv.test-cccnh.svc:18091/node/controller/reloadCertificate","error":"Post \"https://test-couchbase-g52hv-0000.test-couchbase-g52hv.test-cccnh.svc:18091/node/controller/reloadCertificate\": x509: certificate has expired or is not yet valid: current time 2022-08-02T01:14:32Z is after 2022-08-02T01:14:11Z","time_ms":6.7723} |
Attempts to restart the Operator also fail as the first step in the reconcile loop is to get server status which also fails for similar reasons:
{"level":"debug","ts":1659404374.3002732,"logger":"api","msg":"http","cluster":"test-fwjgj/test-couchbase-zrjdq","method":"GET","url":"https://test-couchbase-zrjdq-0000.test-couchbase-zrjdq.test-fwjgj.svc:18091/pools/default","error":"Get \"https://test-couchbase-zrjdq-0000.test-couchbase-zrjdq.test-fwjgj.svc:18091/pools/default\": x509: certificate has expired or is not yet valid: current time 2022-08-02T01:39:34Z is after 2022-08-02T01:24:02Z","time_ms":9.6086} |