Uploaded image for project: 'Couchbase Python Client Library'
  1. Couchbase Python Client Library
  2. PYCBC-445

Implement/test support for KV with homogenous IPv6

    XMLWordPrintable

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.3.2
    • None
    • SDK 45: IPv6, HC, SDK 47: HC, Log Redact, SDK49: HC, Log Reda, CertAuth

    Description

      See CBD for more information.

      This aligns to what we're calling phase one in the CBD. Note that it need not merge into the master branch if there is concern about introducing instability, but does need to be identified on a topic/branch in that case.

      Attachments

        Issue Links

          For Gerrit Dashboard: PYCBC-445
          # Subject Branch Project Status CR V

          Activity

            Ellis.Breen Ellis Breen added a comment - - edited

            Agreed with Matt Ingenthron to provide some test scripts to Jae Park [X] for situational testing if cannot get the Python unit tests 100% tomorrow... Some example test specs would be useful, but general KV engine/N1QL/FTS tests should be fine (Indexing/Query servers not IPV6 ready, yet).

            Ellis.Breen Ellis Breen added a comment - - edited Agreed with Matt Ingenthron to provide some test scripts to Jae Park [X] for situational testing if cannot get the Python unit tests 100% tomorrow... Some example test specs would be useful, but general KV engine/N1QL/FTS tests should be fine (Indexing/Query servers not IPV6 ready, yet).
            Ellis.Breen Ellis Breen added a comment -

            Trying to get the tests to work, getting some odd errors with IPV4/non-mock and couchbaselabs/vagrants/ubuntu14 (CB5.0.1), libcouchbase from the HEAD (with IPV6 changes/authority fixes):

            PYCBC-445.pcapng

             

            nosetests -v couchbase.tests.cases.append_t:AppendTest.test_append_binary
            test_append_binary (couchbase.tests.cases.append_t.AppendTest) ... /Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/tests/base.py:282: UserWarning: Using fallback (couchbase module) defaults
              warnings.warn('Using fallback (couchbase module) defaults')
            0ms [I0] {3916/307} [INFO] (instance - L:427) Version=2.8.3_7_g84f70195, Changeset=84f7019536976c683a034c1ffc448c00434f66c3
            0ms [I0] {3916/307} [INFO] (instance - L:428) Effective connection string: http://10.142.171.101:8091/default?ipv6=allow. Bucket=default
            63ms [I0] {3916/307} [INFO] (instance - L:139) DNS SRV lookup failed: DNS/Hostname lookup failed. Ignore this if not relying on DNS SRV records
            64ms [I0] {3916/307} [DEBUG] (instance - L:78) Adding host 10.142.171.101:8091 to initial HTTP bootstrap list
            64ms [I0] {3916/307} [DEBUG] (instance - L:78) Adding host 10.142.171.101:11210 to initial CCCP bootstrap list
            64ms [I0] {3916/307} [TRACE] (instance - L:119) Bootstrap hosts loaded (cccp:1, http:1)
            65ms [I0] {3916/307} [DEBUG] (confmon - L:85) Preparing providers (this may be called multiple times)
            65ms [I0] {3916/307} [DEBUG] (confmon - L:92) Provider CCCP is ENABLED
            65ms [I0] {3916/307} [DEBUG] (confmon - L:92) Provider HTTP is ENABLED
            65ms [I0] {3916/307} [TRACE] (confmon - L:272) Refreshing current cluster map
            65ms [I0] {3916/307} [TRACE] (confmon - L:259) Attempting to retrieve cluster map via CCCP
            65ms [I0] {3916/307} [INFO] (cccp - L:151) Requesting connection to node 10.142.171.101:11210 for CCCP configuration
            65ms [I0] {3916/307} [DEBUG] (lcbio_mgr - L:417) <10.142.171.101:11210> (HE=0x7fc276ed9a10) Creating new connection because none are available in the pool
            65ms [I0] {3916/307} [TRACE] (lcbio_mgr - L:328) <10.142.171.101:11210> (HE=0x7fc276ed9a10) New pool entry: I=0x7fc276eded60
            65ms [I0] {3916/307} [INFO] (connection - L:467) <10.142.171.101:11210> (SOCK=0x7fc276ee1140) Starting. Timeout=2000000us
            66ms [I0] {3916/307} [DEBUG] (connection - L:232) <10.142.171.101:11210> (SOCK=0x7fc276ee1140) Created new socket with FD=5
            67ms [I0] {3916/307} [TRACE] (connection - L:333) <10.142.171.101:11210> (SOCK=0x7fc276ee1140) Scheduling I/O watcher for asynchronous connection completion.
            67ms [I0] {3916/307} [INFO] (connection - L:141) <10.142.171.101:11210> (SOCK=0x7fc276ee1140) Connected established
            67ms [I0] {3916/307} [DEBUG] (connection - L:100) <10.142.171.101:11210> (SOCK=0x7fc276ee1140) Successfully set TCP_NODELAY
            67ms [I0] {3916/307} [DEBUG] (connection - L:100) <10.142.171.101:11210> (SOCK=0x7fc276ee1140) Successfully set TCP_KEEPALIVE
            67ms [I0] {3916/307} [DEBUG] (lcbio_mgr - L:287) <10.142.171.101:11210> (HE=0x7fc276ed9a10) Received result for I=0x7fc276eded60,C=0x0; E=0x0
            67ms [I0] {3916/307} [DEBUG] (lcbio_mgr - L:247) <10.142.171.101:11210> (HE=0x7fc276ed9a10) Assigning R=0x7fc276ead1e0 SOCKET=0x7fc276ee1140
            67ms [I0] {3916/307} [DEBUG] (ioctx - L:99) <10.142.171.101:11210> (CTX=0x7fc279200e20,unknown) Pairing with SOCK=0x7fc276ee1140
            67ms [I0] {3916/307} [DEBUG] (negotiation - L:390) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) HELLO identificator: "libcouchbase/2.8.3_7_g84f70195 (Darwin-16.7.0; x86_64; Clang 9.0.0.9000038) PYCBC/2.3.1.dev5+gcba604b"
            67ms [I0] {3916/307} [DEBUG] (negotiation - L:394) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Request feature: 0x2 (TLS)
            67ms [I0] {3916/307} [DEBUG] (negotiation - L:394) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Request feature: 0x6 (XATTR)
            67ms [I0] {3916/307} [DEBUG] (negotiation - L:394) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Request feature: 0x8 (Select bucket)
            67ms [I0] {3916/307} [DEBUG] (negotiation - L:394) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Request feature: 0x7 (XERROR)
            67ms [I0] {3916/307} [DEBUG] (negotiation - L:394) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Request feature: 0x3 (TCP nodelay)
            68ms [I0] {3916/307} [DEBUG] (negotiation - L:420) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Server supports feature: 0x6 (XATTR)
            68ms [I0] {3916/307} [DEBUG] (negotiation - L:420) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Server supports feature: 0x8 (Select bucket)
            68ms [I0] {3916/307} [DEBUG] (negotiation - L:420) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Server supports feature: 0x7 (XERROR)
            68ms [I0] {3916/307} [DEBUG] (negotiation - L:420) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Server supports feature: 0x3 (TCP nodelay)
            71ms [I0] {3916/307} [INFO] (negotiation - L:474) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Sending SELECT_BUCKET
            72ms [I0] {3916/307} [ERROR] (negotiation - L:152) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl,SASLREQ=0x7fc276d57e00) Error: 0x2, Provided credentials not allowed for bucket
            72ms [I0] {3916/307} [ERROR] (cccp - L:165) <NOHOST:NOPORT> (CTX=0x0,) Could not get configuration: LCB_AUTH_ERROR (0x02)
            72ms [I0] {3916/307} [INFO] (confmon - L:182) Provider 'CCCP' failed
            72ms [I0] {3916/307} [ERROR] (bootstrap - L:170) Failed to bootstrap client=0x7fc276d600b0. Error=LCB_AUTH_ERROR (0x02), Message=No more bootstrap providers remain
            72ms [I0] {3916/307} [DEBUG] (ioctx - L:149) <10.142.171.101:11210> (CTX=0x7fc279200e20,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1
            ERROR
             
            ======================================================================
            ERROR: test_append_binary (couchbase.tests.cases.append_t.AppendTest)
            ----------------------------------------------------------------------
            Traceback (most recent call last):
              File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/tests/base.py", line 369, in setUp
                self.cb = self.make_connection()
              File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/tests/base.py", line 323, in make_connection
                return self.cluster_info.make_connection(self.factory, **kwargs)
              File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/tests/base.py", line 75, in make_connection
                return conncls(**connargs)
              File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/bucket.py", line 251, in __init__
                self._do_ctor_connect()
              File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/bucket.py", line 260, in _do_ctor_connect
                self._connect()
            couchbase.exceptions._AuthError_0x2 (generated, catch AuthError): <RC=0x2[Authentication failed. You may have provided an invalid username/password combination], There was a problem while trying to send/receive your request over the network. This may be a result of a bad network or a misconfigured client or server, C Source=(src/bucket.c,792)>
             
            ----------------------------------------------------------------------
            Ran 1 test in 0.123s

             

            Ellis.Breen Ellis Breen added a comment - Trying to get the tests to work, getting some odd errors with IPV4/non-mock and couchbaselabs/vagrants/ubuntu14 (CB5.0.1), libcouchbase from the HEAD (with IPV6 changes/authority fixes): PYCBC-445.pcapng   nosetests -v couchbase.tests.cases.append_t:AppendTest.test_append_binary test_append_binary (couchbase.tests.cases.append_t.AppendTest) ... /Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client- 2.3 . 1 /couchbase/tests/base.py: 282 : UserWarning: Using fallback (couchbase module) defaults   warnings.warn( 'Using fallback (couchbase module) defaults' ) 0ms [I0] { 3916 / 307 } [INFO] (instance - L: 427 ) Version= 2.8 .3_7_g84f70195, Changeset=84f7019536976c683a034c1ffc448c00434f66c3 0ms [I0] { 3916 / 307 } [INFO] (instance - L: 428 ) Effective connection string: http: //10.142.171.101:8091/default?ipv6=allow. Bucket=default 63ms [I0] { 3916 / 307 } [INFO] (instance - L: 139 ) DNS SRV lookup failed: DNS/Hostname lookup failed. Ignore this if not relying on DNS SRV records 64ms [I0] { 3916 / 307 } [DEBUG] (instance - L: 78 ) Adding host 10.142 . 171.101 : 8091 to initial HTTP bootstrap list 64ms [I0] { 3916 / 307 } [DEBUG] (instance - L: 78 ) Adding host 10.142 . 171.101 : 11210 to initial CCCP bootstrap list 64ms [I0] { 3916 / 307 } [TRACE] (instance - L: 119 ) Bootstrap hosts loaded (cccp: 1 , http: 1 ) 65ms [I0] { 3916 / 307 } [DEBUG] (confmon - L: 85 ) Preparing providers ( this may be called multiple times) 65ms [I0] { 3916 / 307 } [DEBUG] (confmon - L: 92 ) Provider CCCP is ENABLED 65ms [I0] { 3916 / 307 } [DEBUG] (confmon - L: 92 ) Provider HTTP is ENABLED 65ms [I0] { 3916 / 307 } [TRACE] (confmon - L: 272 ) Refreshing current cluster map 65ms [I0] { 3916 / 307 } [TRACE] (confmon - L: 259 ) Attempting to retrieve cluster map via CCCP 65ms [I0] { 3916 / 307 } [INFO] (cccp - L: 151 ) Requesting connection to node 10.142 . 171.101 : 11210 for CCCP configuration 65ms [I0] { 3916 / 307 } [DEBUG] (lcbio_mgr - L: 417 ) < 10.142 . 171.101 : 11210 > (HE= 0x7fc276ed9a10 ) Creating new connection because none are available in the pool 65ms [I0] { 3916 / 307 } [TRACE] (lcbio_mgr - L: 328 ) < 10.142 . 171.101 : 11210 > (HE= 0x7fc276ed9a10 ) New pool entry: I= 0x7fc276eded60 65ms [I0] { 3916 / 307 } [INFO] (connection - L: 467 ) < 10.142 . 171.101 : 11210 > (SOCK= 0x7fc276ee1140 ) Starting. Timeout=2000000us 66ms [I0] { 3916 / 307 } [DEBUG] (connection - L: 232 ) < 10.142 . 171.101 : 11210 > (SOCK= 0x7fc276ee1140 ) Created new socket with FD= 5 67ms [I0] { 3916 / 307 } [TRACE] (connection - L: 333 ) < 10.142 . 171.101 : 11210 > (SOCK= 0x7fc276ee1140 ) Scheduling I/O watcher for asynchronous connection completion. 67ms [I0] { 3916 / 307 } [INFO] (connection - L: 141 ) < 10.142 . 171.101 : 11210 > (SOCK= 0x7fc276ee1140 ) Connected established 67ms [I0] { 3916 / 307 } [DEBUG] (connection - L: 100 ) < 10.142 . 171.101 : 11210 > (SOCK= 0x7fc276ee1140 ) Successfully set TCP_NODELAY 67ms [I0] { 3916 / 307 } [DEBUG] (connection - L: 100 ) < 10.142 . 171.101 : 11210 > (SOCK= 0x7fc276ee1140 ) Successfully set TCP_KEEPALIVE 67ms [I0] { 3916 / 307 } [DEBUG] (lcbio_mgr - L: 287 ) < 10.142 . 171.101 : 11210 > (HE= 0x7fc276ed9a10 ) Received result for I= 0x7fc276eded60 ,C= 0x0 ; E= 0x0 67ms [I0] { 3916 / 307 } [DEBUG] (lcbio_mgr - L: 247 ) < 10.142 . 171.101 : 11210 > (HE= 0x7fc276ed9a10 ) Assigning R= 0x7fc276ead1e0 SOCKET= 0x7fc276ee1140 67ms [I0] { 3916 / 307 } [DEBUG] (ioctx - L: 99 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,unknown) Pairing with SOCK= 0x7fc276ee1140 67ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 390 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) HELLO identificator: "libcouchbase/2.8.3_7_g84f70195 (Darwin-16.7.0; x86_64; Clang 9.0.0.9000038) PYCBC/2.3.1.dev5+gcba604b" 67ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 394 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Request feature: 0x2 (TLS) 67ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 394 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Request feature: 0x6 (XATTR) 67ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 394 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Request feature: 0x8 (Select bucket) 67ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 394 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Request feature: 0x7 (XERROR) 67ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 394 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Request feature: 0x3 (TCP nodelay) 68ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 420 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Server supports feature: 0x6 (XATTR) 68ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 420 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Server supports feature: 0x8 (Select bucket) 68ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 420 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Server supports feature: 0x7 (XERROR) 68ms [I0] { 3916 / 307 } [DEBUG] (negotiation - L: 420 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Server supports feature: 0x3 (TCP nodelay) 71ms [I0] { 3916 / 307 } [INFO] (negotiation - L: 474 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Sending SELECT_BUCKET 72ms [I0] { 3916 / 307 } [ERROR] (negotiation - L: 152 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl,SASLREQ= 0x7fc276d57e00 ) Error: 0x2 , Provided credentials not allowed for bucket 72ms [I0] { 3916 / 307 } [ERROR] (cccp - L: 165 ) <NOHOST:NOPORT> (CTX= 0x0 ,) Could not get configuration: LCB_AUTH_ERROR ( 0x02 ) 72ms [I0] { 3916 / 307 } [INFO] (confmon - L: 182 ) Provider 'CCCP' failed 72ms [I0] { 3916 / 307 } [ERROR] (bootstrap - L: 170 ) Failed to bootstrap client= 0x7fc276d600b0 . Error=LCB_AUTH_ERROR ( 0x02 ), Message=No more bootstrap providers remain 72ms [I0] { 3916 / 307 } [DEBUG] (ioctx - L: 149 ) < 10.142 . 171.101 : 11210 > (CTX= 0x7fc279200e20 ,sasl) Destroying context. Pending Writes= 0 , Entered= true , Socket Refcount= 1 ERROR   ====================================================================== ERROR: test_append_binary (couchbase.tests.cases.append_t.AppendTest) ---------------------------------------------------------------------- Traceback (most recent call last):   File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/tests/base.py" , line 369 , in setUp     self.cb = self.make_connection()   File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/tests/base.py" , line 323 , in make_connection     return self.cluster_info.make_connection(self.factory, **kwargs)   File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/tests/base.py" , line 75 , in make_connection     return conncls(**connargs)   File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/bucket.py" , line 251 , in __init__     self._do_ctor_connect()   File "/Users/ellis_breen/root/workspaces/couchbase/couchbase-python-client-2.3.1/couchbase/bucket.py" , line 260 , in _do_ctor_connect     self._connect() couchbase.exceptions._AuthError_0x2 (generated, catch AuthError): <RC= 0x2 [Authentication failed. You may have provided an invalid username/password combination], There was a problem while trying to send/receive your request over the network. This may be a result of a bad network or a misconfigured client or server, C Source=(src/bucket.c, 792 )>   ---------------------------------------------------------------------- Ran 1 test in 0 .123s  
            Ellis.Breen Ellis Breen added a comment -

            diffs_ipv4_ipv6_interleaved.html - Looks like the test results only differ between the IPV4 and IPV6 in a few, probably expected ways (known non-working systems for IPv6). Just verifying.

            Ellis.Breen Ellis Breen added a comment - diffs_ipv4_ipv6_interleaved.html  - Looks like the test results only differ between the IPV4 and IPV6 in a few, probably expected ways (known non-working systems for IPv6). Just verifying.
            Ellis.Breen Ellis Breen added a comment -

            diffs_notokorskipped_ipv4_ipv6.html - here are just the tests that aren't OK or skipped.

            Ellis.Breen Ellis Breen added a comment - diffs_notokorskipped_ipv4_ipv6.html  - here are just the tests that aren't OK or skipped.
            Ellis.Breen Ellis Breen added a comment - - edited

            diffs_ipv4_ipv6_novieworindex.html - latest test result comparison between IPV4 and IPV6, excluding View/Index which is broken with pure IP-based addressing on IPV6. No apparent regressions.

            Ellis.Breen Ellis Breen added a comment - - edited diffs_ipv4_ipv6_novieworindex.html  - latest test result comparison between IPV4 and IPV6, excluding View/Index which is broken with pure IP-based addressing on IPV6. No apparent regressions.

            People

              Ellis.Breen Ellis Breen
              brett19 Brett Lawson
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty