Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-57385

'WaitUntilReady' returns 'unambiguous timeout' after memcached is killed on all nodes and nodes are back up

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • 7.6.0
    • 7.1.0
    • ns_server
    • Sandbox
    • Untriaged
    • 1
    • Unknown
    • SDK24

    Description

      Test that led to failure/Steps to reproduce:

      1. Created a 3 node provisioned cluster on sandbox and loaded travel-sample bucket.
      2. Killed memcached process on all nodes. Nodes go down for brief period of time when memcached is killed and are back up.
      3. Used the attached main.go file to run some basic operations on the bucket.(Please note that the cluster is in healthy state when I run the file)

      Observation:

      err = cluster.WaitUntilReady(time.Second*240, nil)

       returns 'unambiguous timeout' error. But even though 'WaitUntilReady' for cluster object returns an error. I am able to get the 'bucket' object in the later part of the go file to successfully insert and delete a document from the bucket.
       
      gocb and gocbcore versions:
      github.com/couchbase/gocb/v2 v2.6.3 
      github.com/couchbase/gocbcore/v10 v10.2.3
       
      Cluster logs:
      https://cb-engineering.s3.amazonaws.com/go-sdk-bug/collectinfo-2023-06-13T112603-ns_1%40svc-dqisea-node-001.ssxnmi0dc5xzz-yf.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/go-sdk-bug/collectinfo-2023-06-13T112603-ns_1%40svc-dqisea-node-002.ssxnmi0dc5xzz-yf.sandbox.nonprod-project-avengers.com.zip

      https://cb-engineering.s3.amazonaws.com/go-sdk-bug/collectinfo-2023-06-13T112603-ns_1%40svc-dqisea-node-003.ssxnmi0dc5xzz-yf.sandbox.nonprod-project-avengers.com.zip

      Control plane logs:

      https://app.datadoghq.com/logs?query=container_name%3Acp-jobs%20%40clusterId%3A5a1bc990-be21-45fe-9345-3f7b4b583da4%20&additional_filters=%5B%7B%7D%5D&agg_q=status%2Cservice&cols=host%2Cservice&index=&messageDisplay=inline&sort_m=%2C&sort_t=%2C&stream_sort=time%2Cdesc&top_n=10%2C10&top_o=top%2Ctop&viz=stream&x_missing=true%2Ctrue&from_ts=1682222497803&to_ts=1682308897803&live=true
       

      Attachments

        1. go-sdk.log
          606 kB
        2. main.go
          2 kB

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ashwin.govindarajulu Ashwin Govindarajulu
              mohsin.ahmed Mohsin Ahmed
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty