Uploaded image for project: 'Couchbase Go SDK'
  1. Couchbase Go SDK
  2. GOCBC-890

SearchQueries do not work on non-GCCCP clusters without waiting

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.2
    • Component/s: library
    • Labels:
      None
    • Story Points:
      1

      Description

      Summary

      SearchQuery (FTS) ops do not work in the 2.x Go SDK without sleeping for a while beforehand, if the cluster doesn't support GCCCP (i.e. is <6.5.0).

      Steps to Reproduce

      1. Create a single node 6.0.3 cluster running KV and Search.
      2. Load the travel-sample bucket onto the cluster
      3. Create an FTS index, the default (i.e. just click 'Create Index') is sufficient to demonstrate the issue
      4. Run an FTS search from the Go SDK, ensuring to follow the advice in the documentation:

        When using a Couchbase version < 6.5 you must create a valid Bucket connection using cluster.Bucket(name) before you can use Search.

        I have attached an example self-contained reproduction case as main.go, and the logs as no_sleep.log.

      Expected Results
      The search results are returned successfully.

      Actual Results
      The application panics due to the following error:

      panic: failed to get query provider: not connected to cluster | {"query":{"match":"swanky"}}
      

      Analysis
      This seems to not work after opening a bucket, then immediately performing a search.

      There are 2 ways to get this to work, one is to perform some KV ops on the default collection, which forces the connections to open.
      The other is to sleep the calling goroutine for a bit (e.g. 5 seconds), if you sleep then the search works and you don't even need the default collection to be opened.
      I've attached logs with the sleep as with_sleep.log to demonstrate the difference.

      I suspect there's not appropriate handling within the SearchQuery code to wait for a cluster that's still initializing if the cluster does not support GCCCP.

      Workaround

      Wait until the bucket object is ready to have operations performed against it, this can be done using the Bucket.WaitUntilReady() API.

      For example:

              bucket := cluster.Bucket("travel-sample")
              bucket.WaitUntilReady(30 * time.Second, &gocb.WaitUntilReadyOptions{DesiredState: gocb.ClusterStateOnline})
       
              matchResult, err := cluster.SearchQuery(
                      "travel-sample-index-hotel-description",
                      search.NewMatchQuery("swanky"),
                      &gocb.SearchOptions{
                              Limit: 10,
                      },
              )

        Attachments

        1. with_sleep.log
          23 kB
        2. no_sleep.log
          3 kB
        3. main.go
          1.0 kB

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            charles.dixon Charles Dixon added a comment -

            Hi Matt Carabine this is a known issue which I'm working on resolving to make feel like they behave in a similar way as KV operations. At the moment when a cluster or bucket is opened then we will attempt to asynchronously create connections to the node(s) in the connection string (note: connection time errors are no longer returned). KV operations are actually queued until the underlying connections are ready or the operation times out. The piece of work that I'm doing will make http based services also wait for connections or timeout.

            The workaround for this for now (and it's possible that users may wish to continue to use this) is to use either `cluster.WaitUntilReady` or `bucket.WaitUntilReady`. These operations will currently wait for the underlying memcached connections to be ready for use or timeout. A future enhancement will also improve this to wait for specified services (such as fts) to be ready on the cluster.

            Show
            charles.dixon Charles Dixon added a comment - Hi Matt Carabine this is a known issue which I'm working on resolving to make feel like they behave in a similar way as KV operations. At the moment when a cluster or bucket is opened then we will attempt to asynchronously create connections to the node(s) in the connection string (note: connection time errors are no longer returned). KV operations are actually queued until the underlying connections are ready or the operation times out. The piece of work that I'm doing will make http based services also wait for connections or timeout. The workaround for this for now (and it's possible that users may wish to continue to use this) is to use either `cluster.WaitUntilReady` or `bucket.WaitUntilReady`. These operations will currently wait for the underlying memcached connections to be ready for use or timeout. A future enhancement will also improve this to wait for specified services (such as fts) to be ready on the cluster.
            Hide
            matt.carabine Matt Carabine added a comment -

            I thought that would be the case, but I didn't know about the waitUntilReady method, that's much better than what I'd suggested.

            Maybe would be good to get that included as part of our docs, given the default behavior of how the cluster object and connections are initialized?

            Show
            matt.carabine Matt Carabine added a comment - I thought that would be the case, but I didn't know about the waitUntilReady method, that's much better than what I'd suggested. Maybe would be good to get that included as part of our docs, given the default behavior of how the cluster object and connections are initialized?
            Hide
            charles.dixon Charles Dixon added a comment -

            I agree, I believe that it's on the roadmap to be documented. It's in a slightly weird place where it shouldn't be necessary once the fix is in place but is necessary at the moment. I will also add it as a known issue in the release notes including the above workaround.

            Show
            charles.dixon Charles Dixon added a comment - I agree, I believe that it's on the roadmap to be documented. It's in a slightly weird place where it shouldn't be necessary once the fix is in place but is necessary at the moment. I will also add it as a known issue in the release notes including the above workaround.
            Hide
            matt.carabine Matt Carabine added a comment -

            I've updated the workaround in my issue description too, to encourage its use rather than my hacky retry

            Show
            matt.carabine Matt Carabine added a comment - I've updated the workaround in my issue description too, to encourage its use rather than my hacky retry
            Hide
            charles.dixon Charles Dixon added a comment -

            This behaviour is now that the SDK will "queue" (it doesn't actually queue, it creates a goroutine and spins it with a short (500 micros) sleep) any HTTP based requests (such as search) until the SDK has seen a cluster map from the server or the request times out. This makes usage in line with KV.

            Documentation for gocb now has WaitUntilReady in every code example that shows opening a bucket or cluster also.

            Show
            charles.dixon Charles Dixon added a comment - This behaviour is now that the SDK will "queue" (it doesn't actually queue, it creates a goroutine and spins it with a short (500 micros) sleep) any HTTP based requests (such as search) until the SDK has seen a cluster map from the server or the request times out. This makes usage in line with KV. Documentation for gocb now has WaitUntilReady in every code example that shows opening a bucket or cluster also.

              People

              Assignee:
              charles.dixon Charles Dixon
              Reporter:
              matt.carabine Matt Carabine
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty