Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-42637

Secure (https) Analytics S3 Remote Links are broken when node-to-node encryption is enabled

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Story Points:
      1
    • Is this a Regression?:
      No
    • Sprint:
      CX Sprint 225

      Description

      Summary
      When node-to-node encryption is enabled within the Couchbase Server cluster, all queries using a secure service endpoint (the default) to an s3 link fail.

      Steps to Reproduce

      • Create cluster with a single Analytics + Data node
      • Disable automatic failover:

        /opt/couchbase/bin/couchbase-cli setting-autofailover -c localhost -u Administrator -p password --enable-auto-failover 0
        

      • Enable node-to-node encryption:

        /opt/couchbase/bin/couchbase-cli node-to-node-encryption -c localhost -u Administrator -p password --enable
        

      • Set encryption level to 'all' (unsure if this is required):

        /opt/couchbase/bin/couchbase-cli -c localhost -u Administrator -p password --cluster-encryption-level all
        

      • Create an external s3 link (you don't need actual working creds, you can just copy paste this):

        curl -u Administrator:password -X POST "http://localhost:8095/analytics/link" -d dataverse=Default -d name=s3Link -d type=S3 -d accessKeyId=abcd --data-urlencode secretAccessKey=abcd-d region=us-east-2
        

      • Try to create an external dataset using this s3 link:

        CREATE EXTERNAL DATASET S3productreviews
        ON `cbc-remote-links-test`
        AT s3Link
        USING "reviews"
        WITH { "format": "json", "include": "*.json" } ;
        

      Expected Behavior
      Dataset is created (assuming correct keys etc), at least an error about AWS auth if not quite correct.

      Actual Behavior
      The query returns the error:

      [
        {
          "code": 24086,
          "msg": "External source error. Unable to execute HTTP request: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
          "query_from_user": "CREATE EXTERNAL DATASET S3productreviews5\nON `cbc-remote-links-test`\nAT mattS3LinkEnd\nUSING \"reviews\"\nWITH { \"format\": \"json\", \"include\": \"*.json\" } ;"
        }
      ]
      


      This is because it cannot validate the certificate returned by the S3 endpoint due to trust store issues, likely as a result of some changes applied to trusting the server root CA for node to node encryption.

      Workaround
      Do not use HTTPS for the s3 connection by overriding the serviceEndpoint when creating the link to be http://s3.amazonaws.com, do note that all data retrieved from S3 will go over the network unencrypted.

      e.g.:

      curl -u Administrator:password -X POST "http://localhost:8095/analytics/link" -d dataverse=Default -d name=s3Link -d type=S3 -d accessKeyId=abcd --data-urlencode secretAccessKey=abcd-d region=us-east-2 -d serviceEndpoint=http://s3.amazonaws.com
      

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            There are no commits yet on this issue.

              People

              Assignee:
              umang.agrawal Umang
              Reporter:
              matt.carabine Matt Carabine
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty