Details

    • Bug
    • Resolution: Unresolved
    • Critical
    • None
    • None
    • None
    • None
    • 1

    Description

      7.0.0 Build 3530
      python SDK 2.5.12
      libcouchbase 2.10.7

      Having some real trouble getting indexes to build on 7.0 from the python SDK. It seems to build some but then get stuck and timeout:

      app.supportal              | INFO:couchbase.bootstrap:[2193036225] Requested network configuration: heuristic (L:218)
      app.supportal              | INFO:couchbase.cccp:[2193036225] Requesting connection to node 172.23.97.86:11210 for CCCP configuration (L:151)
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.86:11210> (SOCK=4ea098ad5150bfc1) Starting. Timeout=2000000us (L:474)
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.86:11210> (SOCK=4ea098ad5150bfc1) Connected established (L:147)
      app.supportal              | INFO:couchbase.lcbio_mgr:[2193036225] <172.23.97.86:11210> (HE=0x564dfadb0170) Placing socket back into the pool. I=0x564dfcc8cd20,C=0x564dfab57a90 (L:498)
      app.supportal              | INFO:couchbase.confmon:[2193036225] Setting new configuration. Received via CCCP (L:160)
      app.supportal              | INFO:couchbase.bootstrap:[2193036225] Selected network configuration: "default" (L:90)
      app.supportal              | /supportal/core/indexes.py:33: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
      app.supportal              |   indexes = load(stream)
      app.supportal              | Building Index: CREATE INDEX zendesk_ticket_bugs ON `supportal`(meta().id, id, fields.`Bug ID`, organization, subject, created_at) WHERE meta().id LIKE "Zendesk/tickets::%" AND fields.`Bug ID` IS NOT NULL WITH {"num_replica": 1}
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.85:8093> (SOCK=b7f00965ed034f2a) Starting. Timeout=75000000us (L:474)
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.85:8093> (SOCK=b7f00965ed034f2a) Connected established (L:147)
      app.supportal              | INFO:couchbase.lcbio_mgr:[2193036225] <172.23.97.85:8093> (HE=0x564dfaea8330) Placing socket back into the pool. I=0x564dfcbc3470,C=0x564dfc6e3a60 (L:498)
      app.supportal              | Skipping index as it already exists
      app.supportal              | Building Index: CREATE INDEX account_names ON `supportal`(account_name, cb_account_number) WITH {"num_replica": 1}
      app.supportal              | INFO:couchbase.lcbio_mgr:[2193036225] <172.23.97.85:8093> (HE=0x564dfaea8330) Found ready connection in pool. Reusing socket and not creating new connection (L:441)
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.84:11210> (SOCK=e0335fe17ef54689) Starting. Timeout=2500000us (L:474)
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.84:11210> (SOCK=e0335fe17ef54689) Connected established (L:147)
      app.supportal              | INFO:couchbase.lcbio_mgr:[2193036225] <172.23.97.85:8093> (HE=0x564dfaea8330) Placing socket back into the pool. I=0x564dfcbc3470,C=0x564dfc6e3a60 (L:498)
      app.supportal              | Skipping index as it already exists
      app.supportal              | Building Index: CREATE INDEX adv_account_name_cb_account_number ON `supportal`(lower(account_name), cb_account_number) WITH {"num_replica": 1}
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.84:8093> (SOCK=7ed21639ebe961e3) Starting. Timeout=75000000us (L:474)
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.84:8093> (SOCK=7ed21639ebe961e3) Connected established (L:147)
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.85:11210> (SOCK=821e9ddcb151d211) Starting. Timeout=2500000us (L:474)
      app.supportal              | INFO:couchbase.connection:[2193036225] <172.23.97.85:11210> (SOCK=821e9ddcb151d211) Connected established (L:147)
      app.supportal              | INFO:couchbase.lcbio_mgr:[2193036225] <172.23.97.86:11210> (HE=0x564dfadb0170) Found ready connection in pool. Reusing socket and not creating new connection (L:441)
      app.supportal              | Traceback (most recent call last):
      app.supportal              |   File "/supportal/start.py", line 53, in <module>
      app.supportal              |     build_indexes()
      app.supportal              |   File "/supportal/core/indexes.py", line 38, in build_indexes
      app.supportal              |     build_index(bucket, index, num_replicas=1)
      app.supportal              |   File "/supportal/core/indexes.py", line 54, in build_index
      app.supportal              |     bucket.n1ql_query(statement).execute()
      app.supportal              |   File "/usr/local/lib/python2.7/site-packages/couchbase/n1ql.py", line 496, in execute
      app.supportal              |     for _ in self:
      app.supportal              |   File "/usr/local/lib/python2.7/site-packages/couchbase/n1ql.py", line 523, in __iter__
      app.supportal              |     raw_rows = self.raw.fetch(self._mres)
      app.supportal              | couchbase.exceptions._TimeoutError_0x17 (generated, catch TimeoutError): <RC=0x17[Client-Side timeout exceeded for operation. Inspect network conditions or increase the timeout], HTTP Request failed. Examine 'objextra' for full result, Results=1, C Source=(src/http.c,144), OBJ=ViewResult<rc=0x17[Client-Side timeout exceeded for operation. Inspect network conditions or increase the timeout], value=None, http_status=0, tracing_context=0, tracing_output=None>, Tracing Output={":nokey:0": null}>
      app.supportal              | Traceback (most recent call last):
      app.supportal              |   File "/usr/local/lib/python2.7/site-packages/txcouchbase/iops.py", line 134, in timer_event_factory
      app.supportal              |     return TxTimer()
      app.supportal              | TypeError: 'NoneType' object is not callable
      app.supportal              | Aborted
      app.supportal exited with code 134
      

      When I try to create the index on the server-side it completes in a reasonable timeframe.

      MIght be something in libcouchbase? Sergey Avseyev

      Server-logs are here:
      https://s3.amazonaws.com/cb-engineering/perry/index_build_timeouts/collectinfo-2020-10-29T133818-ns_1%40172.23.97.84.zip
      https://s3.amazonaws.com/cb-engineering/perry/index_build_timeouts/collectinfo-2020-10-29T133818-ns_1%40172.23.97.85.zip
      https://s3.amazonaws.com/cb-engineering/perry/index_build_timeouts/collectinfo-2020-10-29T133818-ns_1%40172.23.97.86.zip

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            jared.casey Jared Casey
            perry Perry Krug
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty