Passing an X509CertificateFactory breaks Http services (Auth failure)

Description

Motivation

When passing a custom X509CertificateFactory to the ClusterOptions, Queries stop working and fail with:

(Note: This seems to be affecting other HTTP services, not just Query. User management operations throw HTTP 401 - Unauthorized)

Unhandled exception. Couchbase.CouchbaseException: Error authorizing against cluster cause: Failure to authenticate user [2120]
   at Couchbase.Query.QueryResultExtensions.ThrowExceptionOnError[T](IQueryResult`1 result, QueryErrorContext context)
   at Couchbase.Query.QueryClient.ExecuteQuery[T](QueryOptions options, ITypeSerializer serializer, IRequestSpan span)
   at Couchbase.Query.QueryClient.QueryAsync[T](String statement, QueryOptions options)
   at Couchbase.Cluster.<>c__DisplayClass37_0`1.<<QueryAsync>g__Func|0>d.MoveNext()

The specific line causing this issue is:

clusterOptions.X509CertificateFactory = CertificateFactory.FromCertificates(certificate);

Not adding a custom X509CertificateFactory, and instead only passing in custom CertificateCallbackValidation for both KV and Http services produces the correct and expected behaviour. (i.e. If the certificate is correct, operations succeed. If the certificate is incorrect, they don't.)

Code to replicate:

public class Program
{
    internal static readonly string WrongCert =
@"-----BEGIN CERTIFICATE-----
MIIDDDCCAfSgAwIBAgIIF7nmQD3WCgQwDQYJKoZIhvcNAQELBQAwJDEiMCAGA1UE
AxMZQ291Y2hiYXNlIFNlcnZlciBjMzcxMTYzMzAeFw0xMzAxMDEwMDAwMDBaFw00
OTEyMzEyMzU5NTlaMCQxIjAgBgNVBAMTGUNvdWNoYmFzZSBTZXJ2ZXIgYzM3MTE2
MzMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC9UTImtOak5Co/r53s
GThiybcVCylrSGCwtqhYn26wzU7TslbxQizMEoMGF17EN5jh1XXFZPr0A9Ss7rcT
QWoRYf9JeiQm8c1CHRdrwo2d9ged/sg754uKNugpuSq9MlPcOCj6V/jtmWJvRNCa
jHEbg236SJac80lUw+0TvqvgZBlukqrUKiPf6omeHRDHIvaGik5Fad1P8MiQ4LdL
/N97MdyFoDZGjFhA/8JFOoSuEZGgw1bP+2ptiU6LEetfwXADokyWZIxG9cMj5lfr
/GuIBRM6B7o9Uf1KhttDfAprHp+sENSO3pFFpYI+s/nagyaqoBJjK2onub7z6XyR
wtutAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBhjAPBgNVHRMBAf8EBTADAQH/MB0G
A1UdDgQWBBSqOAxnXIOVlhM56XakY7YJicI3izANBgkqhkiG9w0BAQsFAAOCAQEA
ZrNj6SiXH+mgOiBbjBMSdLe3Wkc4LKaOHL9bOmz1CYktERV7iXqPTa1E2LzWpBBL
X68AhADDGOLmKhhcI/ka81E66bObKtf5pDQ58HeEv9XhQwkMBucAXTudNtK1PHB5
9+MJq/SMYIBswmFy1jVB34FpE3PYQ/hw81G5dtckan/tDhsP+Ur91zlvIWHZA1my
ZvllpxZx6NG8sqdY64J7/KCYiTTSshtI6lpQNsYKn3qEEcNVU5tCiM8IUCwe+v7p
m5hAMgGnbsjWapDYPONrL1QAnpzAIuG8U4zEdKJy6AoQ7KrSbTckDyGxT06ZPcKq
KzsBw2nk6zu9eSH1vtjM7w==
-----END CERTIFICATE-----";

    internal static readonly string CapellaCaCertPem =
        @"-----BEGIN CERTIFICATE-----
MIIDFTCCAf2gAwIBAgIRANLVkgOvtaXiQJi0V6qeNtswDQYJKoZIhvcNAQELBQAw
JDESMBAGA1UECgwJQ291Y2hiYXNlMQ4wDAYDVQQLDAVDbG91ZDAeFw0xOTEyMDYy
MjEyNTlaFw0yOTEyMDYyMzEyNTlaMCQxEjAQBgNVBAoMCUNvdWNoYmFzZTEOMAwG
A1UECwwFQ2xvdWQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCfvOIi
enG4Dp+hJu9asdxEMRmH70hDyMXv5ZjBhbo39a42QwR59y/rC/sahLLQuNwqif85
Fod1DkqgO6Ng3vecSAwyYVkj5NKdycQu5tzsZkghlpSDAyI0xlIPSQjoORA/pCOU
WOpymA9dOjC1bo6rDyw0yWP2nFAI/KA4Z806XeqLREuB7292UnSsgFs4/5lqeil6
rL3ooAw/i0uxr/TQSaxi1l8t4iMt4/gU+W52+8Yol0JbXBTFX6itg62ppb/Eugmn
mQRMgL67ccZs7cJ9/A0wlXencX2ohZQOR3mtknfol3FH4+glQFn27Q4xBCzVkY9j
KQ20T1LgmGSngBInAgMBAAGjQjBAMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYE
FJQOBPvrkU2In1Sjoxt97Xy8+cKNMA4GA1UdDwEB/wQEAwIBhjANBgkqhkiG9w0B
AQsFAAOCAQEARgM6XwcXPLSpFdSf0w8PtpNGehmdWijPM3wHb7WZiS47iNen3oq8
m2mm6V3Z57wbboPpfI+VEzbhiDcFfVnK1CXMC0tkF3fnOG1BDDvwt4jU95vBiNjY
xdzlTP/Z+qr0cnVbGBSZ+fbXstSiRaaAVcqQyv3BRvBadKBkCyPwo+7svQnScQ5P
Js7HEHKVms5tZTgKIw1fbmgR2XHleah1AcANB+MAPBCcTgqurqr5G7W2aPSBLLGA
fRIiVzm7VFLc7kWbp7ENH39HVG6TZzKnfl9zJYeiklo5vQQhGSMhzBsO70z4RRzi
DPFAN/4qZAgD5q3AFNIq2WWADFQGSwVJhg==
-----END CERTIFICATE-----";

    public static async Task Main(string[] args)
    {
        var clusterOptions = new ClusterOptions
        {
            ConnectionString = "couchbases://your-capella-hostname",
            UserName = "Administrator",
            Password = "Password123!"
        };

        var certificate = new X509Certificate2(
            rawData: System.Text.Encoding.ASCII.GetBytes(CapellaCaCertPem),
            password: (string)null!);
        clusterOptions.X509CertificateFactory = CertificateFactory.FromCertificates(certificate);

        var x509CertCollection = new X509Certificate2Collection(certificate);

        RemoteCertificateValidationCallback certificateValidationCallback = GetValidatorWithPredefinedCertificates(x509CertCollection);
        clusterOptions.KvCertificateCallbackValidation = certificateValidationCallback;
        clusterOptions.HttpCertificateCallbackValidation = certificateValidationCallback;

        
        //Query Test
        var cluster = await Cluster.ConnectAsync(clusterOptions).ConfigureAwait(false);
        var result = await cluster.QueryAsync<dynamic>("SELECT * from `default` limit 10;").ConfigureAwait(false);
        await foreach (var el in result.Rows)
        {
            Console.WriteLine(el.ToString());
        }
        
        //KV Test
        var bucket = await cluster.BucketAsync("default").ConfigureAwait(false);
        var scope = await bucket.DefaultScopeAsync().ConfigureAwait(false);
        var collection = await scope.CollectionAsync("_default").ConfigureAwait(false);
        
        await collection.UpsertAsync("myDoc", new {Content = "MyContent"}).ConfigureAwait(false);
        var myDoc = await collection.GetAsync("myDoc").ConfigureAwait(false);
        Console.WriteLine(myDoc.ContentAs<dynamic>());

    }

    private static RemoteCertificateValidationCallback GetValidatorWithPredefinedCertificates(X509Certificate2Collection certs) =>
        (object sender, X509Certificate? certificate, X509Chain? chain, SslPolicyErrors sslPolicyErrors) =>
        {
            if (sslPolicyErrors == SslPolicyErrors.None)
            {
                return true;
            }

            if (chain == null)
            {
                return false;
            }

#if NET5_0_OR_GREATER
            chain.ChainPolicy.TrustMode = X509ChainTrustMode.CustomRootTrust;
            foreach (var defaultCert in certs)
            {
                chain.ChainPolicy.CustomTrustStore.Add(defaultCert);
            }
#endif
            if (certificate is X509Certificate2 cert2)
            {
                chain.Reset();
                var built = chain.Build(cert2);
                return built;
            }
            return false;
        };
}

Environment

None

Gerrit Reviews

None

Release Notes Description

None

Linked issues

relates

NCBC-3272

Use Platform Root CA by Default

NCBC-3173

Capella CA Certificate Bundled with SDK (Note: fix is .NET 5+ only)

Activity

Show:

Ray Cardillo April 23, 2024 at 2:31 PM
Edited

I think this issue needs some attention. We should "vet" the logic we currently have in docs:

https://docs.couchbase.com/dotnet-sdk/current/howtos/managing-connections.html#couchbase-server

Also, IIRC, we're not supposed to be using a default when provided a cert because the user is indicating they want us to use a specific cert and we don't want the default being used in that case. So KV should not succeed with an incorrect cert.

I linked the related issues we previously worked on in this area for reference.

Emilien Bevierre April 16, 2024 at 1:14 PM
Edited

Using only clusterOptions.X509CertificateFactory = CertificateFactory.FromCertificates(certificate)

Figure 1:

	Correct Cert	Incorrect Cert
KV Ops	✅	✅
Http Ops	❌	❌

Using only custom clusterOptions.KvCertificateCallbackValidation and clusterOptions.HttpCertificateCallbackValidation

Figure 2:

	Correct Cert	Incorrect Cert
KV Ops	✅	❌
Http Ops	✅	❌

Also, by changing the pre-packaged Capella cert in the SDK to an incorrect certificate, Figure 1 then looks like:

Figure 3:

	Correct Cert	Incorrect Cert
KV Ops	❌	❌
Http Ops	❌	❌

# Is using clusterOptions.X509CertificateFactory the "normal" correct way to add custom certificates to the client for it to authenticate itself?

For KV: It seems like we're not using the custom certs at all for KV using X509CertificateFactory, and we're just using the pre-packaged Capella one in the SDK. (Figure 1 and Figure 3)
For Http: It seems passing in a custom X509CertificateFactory overwrites the pre-packaged Capella Certificate, yet does not use the new certificates correctly (Figure 1).

Jeffry Morris April 15, 2024 at 7:48 PM

Based on the fact that the CertificateFactory.FromCertificates creates a PredefinedCertificateFactory which creates a X509Certificate2Collection makes me think the issue is directly related to passed in certificate.

Fixed

Pinned fields

Click on the next to a field label to start pinning.

Details

Assignee

Emilien Bevierre

Reporter

Emilien Bevierre

Story Points

Sprint

None

Fix versions

3.6.0

Priority

Major

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created April 15, 2024 at 8:29 AM

Updated May 1, 2024 at 4:06 PM

Resolved May 1, 2024 at 4:06 PM

Configure

Instabug

Passing an X509CertificateFactory breaks Http services (Auth failure)

Description

Motivation

Environment

Gerrit Reviews

Release Notes Description

Linked issues

relates

Activity

Ray Cardillo April 23, 2024 at 2:31 PMEdited

Emilien Bevierre April 16, 2024 at 1:14 PMEdited

Figure 1:

Figure 2:

Figure 3:

Jeffry Morris April 15, 2024 at 7:48 PM

Details

Assignee

Reporter

Story Points

Sprint

Fix versions

Priority

Instabug

PagerDuty

Sentry

Zendesk Support

Ray Cardillo April 23, 2024 at 2:31 PM
Edited

Emilien Bevierre April 16, 2024 at 1:14 PM
Edited