Uploaded image for project: 'Couchbase Python Client Library'
  1. Couchbase Python Client Library
  2. PYCBC-732

QueryIndexManager interface work, plus failing tests

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-beta.1
    • 3.0.0-rc
    • tests
    • None
    • 1
    • SDK7: Devguide Antorized, Docs

    Description

      When all the tests are run, all 4 fail with timeout errors trying to create an index. Run in isolation (just that file), I see:

      ======================================================================
      ERROR: test_alt_indexes (couchbase.tests_v3.cases.querymgmt_t.IndexManagementTestCase)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/tests_v3/cases/querymgmt_t.py", line 110, in test_alt_indexes
          mgr.create_index(bucket_name, ixname, fields=['hello'], ignore_if_exists=True)
        File "<boltons.funcutils.FunctionBuilder-32>", line 2, in create_index
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/exceptions.py", line 1007, in wrapped
          return func(*args, **kwargs)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 197, in create_index
          self._create_index(bucket_name, fields, index_name, **kwargs)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 203, in _create_index
          self._n1ql_index_create(bucket_name, index_name, fields=fields, **final_args)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 149, in _n1ql_index_create
          return IxmgmtRequest(self._admin_bucket, 'create', info, **options).execute()
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 160, in execute
          return [x for x in self]
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 160, in <listcomp>
          return [x for x in self]
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 147, in __iter__
          raw_rows = self.__raw.fetch(self._mres)
      couchbase_core.exceptions.CouchbaseError: <RC=0xDB[LCB_ERR_INDEX_EXISTS (219)], HTTP Request failed. Examine 'objextra' for full result, Results=1, C Source=(src/pycbc_http.c,156), OBJ=ViewResult<rc=0xDB[LCB_ERR_INDEX_EXISTS (219)], value={'requestID': '5210b6a8-e7cd-474d-828f-6404058d2e33', 'clientContextID': '8387e86b59da5524', 'errors': [{'code': 4300, 'msg': 'The index ix2 already exists.'}], 'status': 'fatal', 'metrics': {'elapsedTime': '6.896477ms', 'executionTime': '6.857189ms', 'resultCount': 0, 'resultSize': 0, 'errorCount': 1}}, http_status=0, tracing_context=0, tracing_output=None>, Tracing Output={":nokey:0": null}>
      -------------------- >> begin captured logging << --------------------
      root: INFO: info is {'host': '10.143.195.101', 'port': 8091, 'admin_username': 'Administrator', 'admin_password': 'password', 'bucket_name': 'default', 'bucket_password': 'password', 'ipv6': 'disabled', 'protocol': 'http', 'enable_tracing': None, 'tracingparms': {'port': None}, 'bucket_username': None, 'certpath': None, 'keypath': None, 'analytics_host': '10.143.195.101', 'analytics_port': 8091, 'network': None}
      --------------------- >> end captured logging << ---------------------
       
      ======================================================================
      ERROR: test_create_primary (couchbase.tests_v3.cases.querymgmt_t.IndexManagementTestCase)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/tests_v3/cases/querymgmt_t.py", line 72, in test_create_primary
          mgr.create_primary_index(bucket_name, ignore_if_exists=True)
        File "<boltons.funcutils.FunctionBuilder-33>", line 2, in create_primary_index
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/exceptions.py", line 1007, in wrapped
          return func(*args, **kwargs)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 237, in create_primary_index
          self._create_index(bucket_name, "", [], primary=True, *options, **kwargs)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 203, in _create_index
          self._n1ql_index_create(bucket_name, index_name, fields=fields, **final_args)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 149, in _n1ql_index_create
          return IxmgmtRequest(self._admin_bucket, 'create', info, **options).execute()
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 160, in execute
          return [x for x in self]
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 160, in <listcomp>
          return [x for x in self]
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 147, in __iter__
          raw_rows = self.__raw.fetch(self._mres)
      couchbase_core.exceptions.CouchbaseError: <RC=0xDB[LCB_ERR_INDEX_EXISTS (219)], HTTP Request failed. Examine 'objextra' for full result, Results=1, C Source=(src/pycbc_http.c,156), OBJ=ViewResult<rc=0xDB[LCB_ERR_INDEX_EXISTS (219)], value={'requestID': '4fb549ae-2e94-4916-b715-fb4aeced6139', 'clientContextID': '468eaa5ff07dde95', 'signature': None, 'results': [], 'errors': [{'code': 5000, 'msg': 'GSI CreatePrimaryIndex() - cause: Index #primary already exists.'}], 'status': 'errors', 'metrics': {'elapsedTime': '2.183054ms', 'executionTime': '2.143933ms', 'resultCount': 0, 'resultSize': 0, 'errorCount': 1}}, http_status=0, tracing_context=0, tracing_output=None>, Tracing Output={":nokey:0": null}>
       
      ----------------------------------------------------------------------
      Ran 4 tests in 19.103s
       
      FAILED (errors=2)
      

      Against 6.0, I see:

      ======================================================================
      ERROR: test_alt_indexes (couchbase.tests_v3.cases.querymgmt_t.IndexManagementTestCase)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/tests_v3/cases/querymgmt_t.py", line 110, in test_alt_indexes
          mgr.create_index(bucket_name, ixname, fields=['hello'], ignore_if_exists=True)
        File "<boltons.funcutils.FunctionBuilder-32>", line 2, in create_index
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/exceptions.py", line 1007, in wrapped
          return func(*args, **kwargs)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 197, in create_index
          self._create_index(bucket_name, fields, index_name, **kwargs)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 203, in _create_index
          self._n1ql_index_create(bucket_name, index_name, fields=fields, **final_args)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 149, in _n1ql_index_create
          return IxmgmtRequest(self._admin_bucket, 'create', info, **options).execute()
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 160, in execute
          return [x for x in self]
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 160, in <listcomp>
          return [x for x in self]
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 147, in __iter__
          raw_rows = self.__raw.fetch(self._mres)
      couchbase_core.exceptions.CouchbaseError: <RC=0xDB[LCB_ERR_INDEX_EXISTS (219)], HTTP Request failed. Examine 'objextra' for full result, Results=1, C Source=(src/pycbc_http.c,156), OBJ=ViewResult<rc=0xDB[LCB_ERR_INDEX_EXISTS (219)], value={'requestID': '51eb4edf-f2d3-4e76-b6d9-9bb8bef2688f', 'clientContextID': '3d5237c7655e5fea', 'errors': [{'code': 4300, 'msg': 'The index ix2 already exists.'}], 'status': 'fatal', 'metrics': {'elapsedTime': '654.25µs', 'executionTime': '581.376µs', 'resultCount': 0, 'resultSize': 0, 'errorCount': 1}}, http_status=0, tracing_context=0, tracing_output=None>, Tracing Output={":nokey:0": null}>
      -------------------- >> begin captured logging << --------------------
      root: INFO: info is {'host': 'localhost', 'port': 8091, 'admin_username': 'Administrator', 'admin_password': 'password', 'bucket_name': 'default', 'bucket_password': 'password', 'ipv6': 'disabled', 'protocol': 'http', 'enable_tracing': None, 'tracingparms': {'port': None}, 'bucket_username': None, 'certpath': None, 'keypath': None, 'analytics_host': 'localhost', 'analytics_port': 8091, 'network': None}
      --------------------- >> end captured logging << ---------------------
       
      ======================================================================
      ERROR: test_create_primary (couchbase.tests_v3.cases.querymgmt_t.IndexManagementTestCase)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/tests_v3/cases/querymgmt_t.py", line 72, in test_create_primary
          mgr.create_primary_index(bucket_name, ignore_if_exists=True)
        File "<boltons.funcutils.FunctionBuilder-33>", line 2, in create_primary_index
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/exceptions.py", line 1007, in wrapped
          return func(*args, **kwargs)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 237, in create_primary_index
          self._create_index(bucket_name, "", [], primary=True, *options, **kwargs)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 203, in _create_index
          self._n1ql_index_create(bucket_name, index_name, fields=fields, **final_args)
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase/management/queries.py", line 149, in _n1ql_index_create
          return IxmgmtRequest(self._admin_bucket, 'create', info, **options).execute()
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 160, in execute
          return [x for x in self]
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 160, in <listcomp>
          return [x for x in self]
        File "/Users/davidkelly/projects/gerrit/couchbase-python-client/couchbase_core/_ixmgmt.py", line 147, in __iter__
          raw_rows = self.__raw.fetch(self._mres)
      couchbase_core.exceptions.CouchbaseError: <RC=0xDB[LCB_ERR_INDEX_EXISTS (219)], HTTP Request failed. Examine 'objextra' for full result, Results=1, C Source=(src/pycbc_http.c,156), OBJ=ViewResult<rc=0xDB[LCB_ERR_INDEX_EXISTS (219)], value={'requestID': 'd774189c-6b57-4c98-8819-114c09fb3284', 'clientContextID': 'bab1ba3ed5abc6d3', 'signature': None, 'results': [], 'errors': [{'code': 5000, 'msg': 'GSI CreatePrimaryIndex() - cause: Index #primary already exists.'}], 'status': 'errors', 'metrics': {'elapsedTime': '640.138µs', 'executionTime': '549.645µs', 'resultCount': 0, 'resultSize': 0, 'errorCount': 1}}, http_status=0, tracing_context=0, tracing_output=None>, Tracing Output={":nokey:0": null}>
       
      ----------------------------------------------------------------------
      Ran 4 tests in 54.967s
      
      

      Attachments

        For Gerrit Dashboard: PYCBC-732
        # Subject Branch Project Status CR V

        Activity

          david.kelly David Kelly added a comment -

          This appears at first glance to be due to the ignore_if_exists flag is being ignored. Could be it isn't being passed along to lcb? Or lcb isn't passing it along to the server...

          david.kelly David Kelly added a comment - This appears at first glance to be due to the ignore_if_exists flag is being ignored. Could be it isn't being passed along to lcb? Or lcb isn't passing it along to the server...

          Note that it's a "requirement" to INFO log the environmentals upon instance creation. I say requirement because I've expressed this a bunch of times and we did it for SDK 2.x, but I've not verified it's in the foundational sdk-rfc and SDK 3.x.

          Point is, you should be able to see if lcb is receiving the ignore_if_exists, I think, from the instance creation there unless Python is doing something special.

          ingenthr Matt Ingenthron added a comment - Note that it's a "requirement" to INFO log the environmentals upon instance creation. I say requirement because I've expressed this a bunch of times and we did it for SDK 2.x, but I've not verified it's in the foundational sdk-rfc and SDK 3.x. Point is, you should be able to see if lcb is receiving the ignore_if_exists, I think, from the instance creation there unless Python is doing something special.
          david.kelly David Kelly added a comment -

          Matt Ingenthron - I'll check on this shortly. I'm thinking the http request details really should be logged at debug no matter what. The ignore_if_exists flag on the request, though, not sure if that stuff is logged (didn't see it above). Will look – perhaps we can discuss the environment logging next week?

          david.kelly David Kelly added a comment - Matt Ingenthron - I'll check on this shortly. I'm thinking the http request details really should be logged at debug no matter what. The ignore_if_exists flag on the request, though, not sure if that stuff is logged (didn't see it above). Will look – perhaps we can discuss the environment logging next week?
          david.kelly David Kelly added a comment -

          Also when run as part of the entire suite, I see all 4 timeout. I believe that is related - in the sense that probably any timeout we add is equally ignored. Probably the index takes a bit to create is all? This ticket should cover that too.

          david.kelly David Kelly added a comment - Also when run as part of the entire suite, I see all 4 timeout. I believe that is related - in the sense that probably any timeout we add is equally ignored. Probably the index takes a bit to create is all? This ticket should cover that too.
          david.kelly David Kelly added a comment -

          If the server has developer preview on, index creation will always timeout. So... Perhaps we need to only test against <6.5, or 6.5 without dev preview?

          The integration tests create the beer-search FTS index and a primary index on default bucket before turning the preview on. Maybe we need to test against both flavors of 6.5, and make the tests aware of which we. are using (or perhaps we can examine the settings and determine that in the test).

          david.kelly David Kelly added a comment - If the server has developer preview on, index creation will always timeout. So... Perhaps we need to only test against <6.5, or 6.5 without dev preview? The integration tests create the beer-search FTS index and a primary index on default bucket before turning the preview on. Maybe we need to test against both flavors of 6.5, and make the tests aware of which we. are using (or perhaps we can examine the settings and determine that in the test).
          david.kelly David Kelly added a comment -

          The fix here is to look for an environment variable (or 2, if we run with/without dev preview), and we skip all index creation when appropriate.

          david.kelly David Kelly added a comment - The fix here is to look for an environment variable (or 2, if we run with/without dev preview), and we skip all index creation when appropriate.
          david.kelly David Kelly added a comment -

          And after looking at the tests in more detail, I see the interface is off, as well. Lets fix it all.

          david.kelly David Kelly added a comment - And after looking at the tests in more detail, I see the interface is off, as well. Lets fix it all.

          People

            david.kelly David Kelly
            david.kelly David Kelly
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty