Uploaded image for project: 'Couchbase Java Client'
  1. Couchbase Java Client
  2. JCBC-1811

waitUntilReady throws IllegalStateException with reason 'Against pre 6.5 clusters at least a bucket needs to be opened!'

    XMLWordPrintable

Details

    • 1

    Description

      Server build: 7.0.0-5161

      Java-client: 3.1.4

      Scenario:

      1. Tried opening a bucket object with waitUntilReady call.
      2. Initially seen `com.couchbase.client.core.error.UnambiguousTimeoutException` exception (JCBC-1806). So did a retry of bucket connect request after 2mins of sleep.
      3. During this time observed  'java.lang.IllegalStateException' with reason :: "Against pre 6.5 clusters at least a bucket needs to be opened!" on the 7.0.0 server

      Code reference: https://github.com/couchbaselabs/TAF/blob/9defda6d5c32ee4ade01bf0f5169c1fd2d9d17b3/lib/sdk_client3.py#L297

      Test logs:

      2021-05-16 23:15:23,730 | test  | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:310] WaitUntilReady timeout exception count 1
      2021-05-16 23:15:23,730 | test  | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:312] Exception during waitUntilReady: com.couchbase.client.core.error.UnambiguousTimeoutException: WaitUntilReady timed out {"bucket":"default","checkedServices":["KV"],"currentState":"OFFLINE","desiredState":"ONLINE","services":{},"timeoutMs":120000}
      2021-05-16 23:15:23,756 | test  | INFO    | MainThread | [sdk_client3:get_memory_footprint:329] RAM FootPrint: 1493.3203125
      2021-05-16 23:15:23,767 | test  | INFO    | MainThread | [common_lib:sleep:22] Sleep 120 seconds. Reason: sleep before next retry for bucket connection
      2021-05-16 23:17:23,769 | test  | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:310] WaitUntilReady timeout exception count 2
      2021-05-16 23:17:23,769 | test  | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:312] Exception during waitUntilReady: java.lang.IllegalStateException: Against pre 6.5 clusters at least a bucket needs to be opened!
      2021-05-16 23:17:23,792 | test  | INFO    | MainThread | [sdk_client3:get_memory_footprint:329] RAM FootPrint: 1493.3828125
      2021-05-16 23:17:23,799 | test  | INFO    | MainThread | [common_lib:sleep:22] Sleep 120 seconds. Reason: sleep before next retry for bucket connection

      Note: Seen very rarely in functional tests and not enabled debug / info logs when this error was encountered

       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            ashwin.govindarajulu Ashwin Govindarajulu created issue -
            ashwin.govindarajulu Ashwin Govindarajulu made changes -
            Field Original Value New Value
            Description  

             
            {noformat}
            Server build: 7.0.0-5161
            Java-client: 3.1.4{noformat}
            *Scenario*:
             # Tried opening a bucket object with waitUntilReady call.
             # Initially seen `com.couchbase.client.core.error.UnambiguousTimeoutException` exception (JCBC-1806). So did a retry of bucket connect request after 2mins of sleep.
             # During this time observed  '{{_java.lang.IllegalStateException'_}} with reason :: {color:#0747a6}"_Against pre 6.5 clusters at least a bucket needs to be opened!_"{color} on the 7.0.0 server

            *Code reference*: https://github.com/couchbaselabs/TAF/blob/9defda6d5c32ee4ade01bf0f5169c1fd2d9d17b3/lib/sdk_client3.py#L297

            *Test logs:*
            {noformat}
            2021-05-16 23:15:23,730 | test | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:310] WaitUntilReady timeout exception count 1
            2021-05-16 23:15:23,730 | test | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:312] Exception during waitUntilReady: com.couchbase.client.core.error.UnambiguousTimeoutException: WaitUntilReady timed out {"bucket":"default","checkedServices":["KV"],"currentState":"OFFLINE","desiredState":"ONLINE","services":{},"timeoutMs":120000}
            2021-05-16 23:15:23,756 | test | INFO | MainThread | [sdk_client3:get_memory_footprint:329] RAM FootPrint: 1493.3203125
            2021-05-16 23:15:23,767 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 120 seconds. Reason: sleep before next retry for bucket connection
            2021-05-16 23:17:23,769 | test | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:310] WaitUntilReady timeout exception count 2
            2021-05-16 23:17:23,769 | test | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:312] Exception during waitUntilReady: java.lang.IllegalStateException: Against pre 6.5 clusters at least a bucket needs to be opened!
            2021-05-16 23:17:23,792 | test | INFO | MainThread | [sdk_client3:get_memory_footprint:329] RAM FootPrint: 1493.3828125
            2021-05-16 23:17:23,799 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 120 seconds. Reason: sleep before next retry for bucket connection{noformat}
            *Note*: Seen very rarely in functional tests and not enabled debug / info logs when this error was encountered

             
            Server build: 7.0.0-5161

            Java-client: 3.1.4

            *Scenario*:
             # Tried opening a bucket object with waitUntilReady call.
             # Initially seen `com.couchbase.client.core.error.UnambiguousTimeoutException` exception (JCBC-1806). So did a retry of bucket connect request after 2mins of sleep.
             # During this time observed  '{{_java.lang.IllegalStateException'_}} with reason :: {color:#0747a6}"_Against pre 6.5 clusters at least a bucket needs to be opened!_"{color} on the 7.0.0 server

            *Code reference*: [https://github.com/couchbaselabs/TAF/blob/9defda6d5c32ee4ade01bf0f5169c1fd2d9d17b3/lib/sdk_client3.py#L297]

            *Test logs:*
            {noformat}2021-05-16 23:15:23,730 | test | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:310] WaitUntilReady timeout exception count 1
            2021-05-16 23:15:23,730 | test | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:312] Exception during waitUntilReady: com.couchbase.client.core.error.UnambiguousTimeoutException: WaitUntilReady timed out {"bucket":"default","checkedServices":["KV"],"currentState":"OFFLINE","desiredState":"ONLINE","services":{},"timeoutMs":120000}
            2021-05-16 23:15:23,756 | test | INFO | MainThread | [sdk_client3:get_memory_footprint:329] RAM FootPrint: 1493.3203125
            2021-05-16 23:15:23,767 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 120 seconds. Reason: sleep before next retry for bucket connection
            2021-05-16 23:17:23,769 | test | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:310] WaitUntilReady timeout exception count 2
            2021-05-16 23:17:23,769 | test | CRITICAL | MainThread | [sdk_client3:_SDKClient__create_conn:312] Exception during waitUntilReady: java.lang.IllegalStateException: Against pre 6.5 clusters at least a bucket needs to be opened!
            2021-05-16 23:17:23,792 | test | INFO | MainThread | [sdk_client3:get_memory_footprint:329] RAM FootPrint: 1493.3828125
            2021-05-16 23:17:23,799 | test | INFO | MainThread | [common_lib:sleep:22] Sleep 120 seconds. Reason: sleep before next retry for bucket connection{noformat}
            *Note*: Seen very rarely in functional tests and not enabled debug / info logs when this error was encountered

             
            will.broadbelt Will Broadbelt made changes -
            Link This issue relates to JCBC-1809 [ JCBC-1809 ]
            lynn.straus Lynn Straus made changes -
            Labels affects-cc-testing functional-test affects-cc-testing functional-test must-fix-cc

            Ashwin Govindarajulucan you please retry with 3.1.5. it has improvements with retry on bucket open failures, and based on the error I suspect this could be it. Thanks!

            daschl Michael Nitschinger added a comment - Ashwin Govindarajulu can you please retry with 3.1.5. it has improvements with retry on bucket open failures, and based on the error I suspect this could be it. Thanks!
            daschl Michael Nitschinger made changes -
            Assignee Michael Nitschinger [ daschl ] Ashwin Govindarajulu [ ashwin.govindarajulu ]

            Not seeing this issue with 7.0.0-5177 + sdk v3.1.5.

            We can closing this ticket now and reopen once seen again.

            ashwin.govindarajulu Ashwin Govindarajulu added a comment - Not seeing this issue with 7.0.0-5177 + sdk v3.1.5. We can closing this ticket now and reopen once seen again.
            ashwin.govindarajulu Ashwin Govindarajulu made changes -
            Assignee Ashwin Govindarajulu [ ashwin.govindarajulu ] Michael Nitschinger [ daschl ]
            ritam.sharma Ritam Sharma added a comment -

            Michael Nitschinger - After updating to 3.1.5 we don't see the issue.

            ritam.sharma Ritam Sharma added a comment - Michael Nitschinger - After updating to 3.1.5 we don't see the issue.

            thanks for the update!

            daschl Michael Nitschinger added a comment - thanks for the update!
            daschl Michael Nitschinger made changes -
            Resolution Incomplete [ 4 ]
            Status New [ 10003 ] Resolved [ 5 ]
            ritam.sharma Ritam Sharma made changes -
            Resolution Incomplete [ 4 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            ritam.sharma Ritam Sharma made changes -
            Resolution Fixed [ 1 ]
            Status Reopened [ 4 ] Resolved [ 5 ]
            ashwin.govindarajulu Ashwin Govindarajulu made changes -
            Assignee Michael Nitschinger [ daschl ] Ashwin Govindarajulu [ ashwin.govindarajulu ]
            Status Resolved [ 5 ] Closed [ 6 ]

            People

              ashwin.govindarajulu Ashwin Govindarajulu
              ashwin.govindarajulu Ashwin Govindarajulu
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty