Uploaded image for project: 'Couchbase Python Client Library'
  1. Couchbase Python Client Library
  2. PYCBC-948

Provide wheel with OpenSSL or document distribution with OpenSSL available

    XMLWordPrintable

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.1
    • None
    • None
    • None
    • 1
    • SDK26: TxnFunc2,TxnFormer,KS, SDK28: Txn Loop/Rb,PfQueryColl

    Description

      Tested just now, Python on WIndows (Python 3.8 installed from the Windows app store) does not work against Couchbase Cloud. It looks like this is because Windows Python either is not built with OpenSSL or does not expose the OpenSSL symbols to us.

      It exits with this:

      c:\Users\Matt Ingenthron>python testing.py
      Traceback (most recent call last):
        File "testing.py", line 7, in <module>
          cluster = Cluster.connect(
        File "C:\Users\Matt Ingenthron\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\couchbase\cluster.py", line 508, in connect
          return cls(connection_string, options, **kwargs)
        File "C:\Users\Matt Ingenthron\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\couchbase\cluster.py", line 491, in __init__
          super(Cluster, self).__init__(connection_string=str(self.connstr), _conntype=_LCB.LCB_TYPE_CLUSTER, **self._clusteropts)
        File "C:\Users\Matt Ingenthron\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\couchbase_core\client.py", line 141, in __init__
          super(Client, self).__init__(*args, **kwargs)
      couchbase.exceptions.CouchbaseException: <RC=0x405[LCB_ERR_SDK_FEATURE_UNAVAILABLE (1029)], Couldn't create instance. Either bad credentials/hosts/bucket names were passed, or there was an internal error in creating the object, C Source=(src/bucket.c,1116)>
      

      From this (anonymized) script:

      from couchbase.cluster import Cluster, ClusterOptions, ClusterTimeoutOptions
      from couchbase.search import TermQuery
      from couchbase.search import SearchOptions, HighlightStyle
      from couchbase.auth import PasswordAuthenticator, CertAuthenticator
      from datetime import timedelta
      # 10.143.193.101 = 6.0.3 cluster
      cluster = Cluster.connect(
      #    'couchbases://cb.51bbb323-476e-4354-bec8-5f9b0a67d146.dp.cloud.couchbase.com',
      #    'couchbases://cb.51bbb323-476e-4354-bec8-5f9b0a67d146.dp.cloud.couchbase.com?certpath=c:\\Users\\Matt\ Ingenthron\\Desktop\\cloudcert.pem',
          'couchbases://cb.51bbb323-476e-4354-bec8-5f9b0a67d146.dp.cloud.couchbase.com?ssl=no_verify',
          ClusterOptions(PasswordAuthenticator('youarent', 'allowedtoknow'))
      )
      d = cluster.bucket('travel-sample')
      

      You can see we played with a few samples, but the basic issue is as soon as you try to use TLS, even with just a simple "ssl=no_verify", it fails with LCB_ERR_SDK_FEATURE_UNAVAILABLE. At that stage, all lcb would have tried to do is set up some structures from OpenSSL.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Ellis.Breen Ellis Breen added a comment - - edited

            This is currently a WIP - https://issues.couchbase.com/browse/PYCBC-941.

            I've managed to get OpenSSL-linked PYCBC for Windows running in the above, 
            but it seems standard Python Windows distributions do not bundle OpenSSL itself, but they do dynamically link against OpenSSL (a very specific version thereof, as per OpenSSL ABI standards). However, it seems if you build LCB against OpenSSL, it will not run at all without OpenSSL installed.

            I am going to see if there is some option in LCB to dynamically load OpenSSL at runtime if available but fall back to non-SSL operation otherwise.  Sergey Avseyev - is this an option?

            Alternatively, I see a few options for Windows users:

            1. mandate the user installs OpenSSL themselves in order to run PYCBC at all
            2. pull in OpenSSL binaries during installation (e.g. from the `cryptography` library https://cryptography.io/en/latest/installation/ - this seems to be the de facto OpenSSL/crypto library and apparently statically links the right version of OpenSSL). 
            3. provide two variants of PYCBC wheels, one with SSL bundled and enabled, one with SSL disabled. pip allows optional dependencies 'e.g. couchbase[cryptography]' but I'm not sure this would work with wheel packaging. We could do couchbase-nossl for without ssl or vice versa
            4. leave non-SSL users to build couchbase without SSL themselves
            5. the converse (though SSL by default seems more secure)

            I'm not fully versed in the legalities of crypto export etc to know what is best - what has happened with Node etc?

            I can see libcouchbase Windows binaries are provided in OpenSSL and non OpenSSL-linked form.

             

            Ellis.Breen Ellis Breen added a comment - - edited This is currently a WIP - https://issues.couchbase.com/browse/PYCBC-941 . I've managed to get OpenSSL-linked PYCBC for Windows running in the above,  but it seems standard Python Windows distributions do not bundle OpenSSL itself, but they do dynamically link against OpenSSL (a very specific version thereof, as per OpenSSL ABI standards). However, it seems if you build LCB against OpenSSL, it will not run at all without OpenSSL installed. I am going to see if there is some option in LCB to dynamically load OpenSSL at runtime if available but fall back to non-SSL operation otherwise.   Sergey Avseyev  - is this an option? Alternatively, I see a few options for Windows users: mandate the user installs OpenSSL themselves in order to run PYCBC at all pull in OpenSSL binaries during installation (e.g. from the `cryptography` library https://cryptography.io/en/latest/installation/  - this seems to be the de facto OpenSSL/crypto library and apparently statically links the right version of OpenSSL).  provide two variants of PYCBC wheels, one with SSL bundled and enabled, one with SSL disabled. pip allows optional dependencies 'e.g. couchbase [cryptography] ' but I'm not sure this would work with wheel packaging. We could do couchbase-nossl for without ssl or vice versa leave non-SSL users to build couchbase without SSL themselves the converse (though SSL by default seems more secure) I'm not fully versed in the legalities of crypto export etc to know what is best - what has happened with Node etc? I can see libcouchbase Windows binaries are provided in OpenSSL and non OpenSSL-linked form.  

            For PHP SDK we use libcouchbase with OpenSSL, we just require users to put OpenSSL binaries somewhere in the PATH

            This is the easiest way to support OpenSSL, and using it with dlopen would complicate things for libcouchbase.

            If user can obtain OpenSSL binaries, I think he can just use another package for Python SDK.

            avsej Sergey Avseyev added a comment - For PHP SDK we use libcouchbase with OpenSSL, we just require users to put OpenSSL binaries somewhere in the PATH This is the easiest way to support OpenSSL, and using it with dlopen would complicate things for libcouchbase. If user can obtain OpenSSL binaries, I think he can just use another package for Python SDK.
            Ellis.Breen Ellis Breen added a comment - - edited

            Sounds reasonable - option 2/4. Will try to figure out how this can work. I want to sort out release automation ASAP anyway and this is closely related. 

            Ellis.Breen Ellis Breen added a comment - - edited Sounds reasonable - option 2/4. Will try to figure out how this can work. I want to sort out release automation ASAP anyway and this is closely related. 

            People

              Ellis.Breen Ellis Breen
              ingenthr Matt Ingenthron
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty