Uploaded image for project: 'Java Couchbase JVM Core'
  1. Java Couchbase JVM Core
  2. JVMCBC-949

Memcached_bucket: Selecing non-default collection should failfast

    XMLWordPrintable

Details

    • 1

    Description

      Scenario:

      • Create Memcached bucket on 2 node cluster
      • Open SDK client connection for create mc bucket
      • Try to select non-existing scope/collection and perform document insert

      Observation:

      Insert op fails with AmibiguousTimeoutException  followed by "Could not publish Event because the queue is full" messages. After this state, I see this queue-full message event during default_collection operations. (default-collection operation succeeds)

      {'error': 'com.couchbase.client.core.error.AmbiguousTimeoutException: InsertRequest, Reason: TIMEOUT {"cancelled":true,"completed":true,"coreId":"0x8efb008700000001","idempotent":false,"lastChannelId":"8EFB008700000001/00000000548CFD8D","lastDispatchedFrom":"10.112.210.1:55052","lastDispatchedTo":"10.112.210.102:11210","reason":"TIMEOUT","requestId":14,"requestType":"InsertRequest","retried":19,"retryReasons":["ENDPOINT_NOT_WRITABLE","COLLECTION_MAP_REFRESH_IN_PROGRESS"],"service":{"bucket":"default","collection":"test","documentId":"test","opaque":"0x231","scope":"_default","type":"kv"},"timeoutMs":5000,"timings":{"encodingMicros":1153,"totalMicros":5012651}}', 'cas': 0, 'value': {}, 'key': 'test', 'status': False}
      Could not publish Event because the queue is full. CollectionMapRefreshFailedEvent{severity=WARN, category=com.couchbase.config, duration=PT3.089991955S, createdAt=125792097758670, description=Collection Map refresh (CollectionIdentifier{bucket='default', scope=Optional[_default], collection=Optional[test], isDefault=false}) failed: FAILED, context=CoreContext{coreId=0x8efb008700000001}, cause=com.couchbase.client.core.error.RequestCanceledException: GetCollectionIdRequest, Reason: TOO_MANY_REQUESTS_IN_RETRY {"cancelled":true,"completed":true,"coreId":"0x8efb008700000001","idempotent":true,"lastChannelId":"8EFB008700000001/00000000548CFD8D","lastDispatchedFrom":"10.112.210.1:55052","lastDispatchedTo":"10.112.210.102:11210","reason":"TOO_MANY_REQUESTS_IN_RETRY","requestId":69614,"requestType":"GetCollectionIdRequest","retried":2,"retryReasons":["KV_COLLECTION_OUTDATED"],"service":{"bucket":"default","collection":"test","opaque":"0x11211","scope":"_default","type":"kv"},"timeoutMs":10000,"timings":{"dispatchMicros":1997174,"serverMicros":0}}}
      Could not publish Event because the queue is full. CollectionMapRefreshFailedEvent{severity=WARN, category=com.couchbase.config, duration=PT4.048678927S, createdAt=125792097919150, description=Collection Map refresh (CollectionIdentifier{bucket='default', scope=Optional[_default], collection=Optional[test], isDefault=false}) failed: FAILED, context=CoreContext{coreId=0x8efb008700000001}, cause=com.couchbase.client.core.error.RequestCanceledException: GetCollectionIdRequest, Reason: TOO_MANY_REQUESTS_IN_RETRY {"cancelled":true,"completed":true,"coreId":"0x8efb008700000001","idempotent":true,"lastChannelId":"8EFB008700000001/00000000548CFD8D","lastDispatchedFrom":"10.112.210.1:55052","lastDispatchedTo":"10.112.210.102:11210","reason":"TOO_MANY_REQUESTS_IN_RETRY","requestId":35585,"requestType":"GetCollectionIdRequest","retried":5,"retryReasons":["KV_COLLECTION_OUTDATED","ENDPOINT_NOT_WRITABLE"],"service":{"bucket":"default","collection":"test","opaque":"0x8d24","scope":"_default","type":"kv"},"timeoutMs":10000,"timings":{"dispatchMicros":1997331,"serverMicros":0}}}
      Could not publish Event because the queue is full. CollectionMapRefreshFailedEvent{severity=WARN, category=com.couchbase.config, duration=PT4.356471665S, createdAt=125792097919833, description=Collection Map refresh (CollectionIdentifier{bucket='default', scope=Optional[_default], collection=Optional[test], isDefault=false}) failed: FAILED, context=CoreContext{coreId=0x8efb008700000001}, cause=com.couchbase.client.core.error.RequestCanceledException: GetCollectionIdRequest, Reason: SHUTDOWN {"cancelled":true,"completed":true,"coreId":"0x8efb008700000001","idempotent":true,"lastChannelId":"8EFB008700000001/00000000548CFD8D","lastDispatchedFrom":"10.112.210.1:55052","lastDispatchedTo":"10.112.210.102:11210","reason":"SHUTDOWN","requestId":27192,"requestType":"GetCollectionIdRequest","retried":9,"retryReasons":["KV_COLLECTION_OUTDATED","ENDPOINT_NOT_WRITABLE"],"service":{"bucket":"default","collection":"test","opaque":"0x6c5b","scope":"_default","type":"kv"},"timeoutMs":10000,"timings":{"dispatchMicros":995833,"serverMicros":0}}}
      Could not publish Event because the queue is full. CollectionMapRefreshFailedEvent{severity=WARN, category=com.couchbase.config, duration=PT4.356771059S, createdAt=125792098233623, description=Collection Map refresh (CollectionIdentifier{bucket='default', scope=Optional[_default], collection=Optional[test], isDefault=false}) failed: FAILED, context=CoreContext{coreId=0x8efb008700000001}, cause=com.couchbase.client.core.error.RequestCanceledException: GetCollectionIdRequest, Reason: SHUTDOWN {"cancelled":true,"completed":true,"coreId":"0x8efb008700000001","idempotent":true,"lastChannelId":"8EFB008700000001/00000000548CFD8D","lastDispatchedFrom":"10.112.210.1:55052","lastDispatchedTo":"10.112.210.102:11210","reason":"SHUTDOWN","requestId":27193,"requestType":"GetCollectionIdRequest","retried":9,"retryReasons":["KV_COLLECTION_OUTDATED","ENDPOINT_NOT_WRITABLE"],"service":{"bucket":"default","collection":"test","opaque":"0x6c5c","scope":"_default","type":"kv"},"timeoutMs":10000,"timings":{"dispatchMicros":995836,"serverMicros":0}}}
      ...
      ...
      Could not publish Event because the queue is full. ServiceStateChangedEvent{severity=DEBUG, category=com.couchbase.service, duration=PT0S, createdAt=125792108984491, description=Service changed state from CONNECTED to DEGRADED, context=ServiceContext{bucket=default, coreId=0x8efb008700000001, remote=10.112.210.102:11210, type=KV}, cause=null}
      Could not publish Event because the queue is full. CollectionMapRefreshFailedEvent{severity=WARN, category=com.couchbase.config, duration=PT0.069118502S, createdAt=125792108923821, description=Collection Map refresh (CollectionIdentifier{bucket='default', scope=Optional[_default], collection=Optional[test], isDefault=false}) failed: FAILED, context=CoreContext{coreId=0x8efb008700000001}, cause=com.couchbase.client.core.error.RequestCanceledException: GetCollectionIdRequest, Reason: SHUTDOWN {"cancelled":true,"completed":true,"coreId":"0x8efb008700000001","idempotent":true,"lastDispatchedTo":"10.112.210.102","reason":"SHUTDOWN","requestId":220637,"requestType":"GetCollectionIdRequest","retried":2,"retryReasons":["ENDPOINT_NOT_WRITABLE"],"service":{"bucket":"default","collection":"test","opaque":"0x36000","scope":"_default","type":"kv"},"timeoutMs":10000}}
      Could not publish Event because the queue is full. RequestRetryScheduledEvent{severity=DEBUG, category=com.couchbase.request, duration=PT0.5S, createdAt=125792108987382, description=Request GetCollectionIdRequest retry scheduled per RetryStrategy (Reason: KV_COLLECTION_OUTDATED), context=RequestContext{completed=false, coreId=0x8efb008700000001, idempotent=true, lastChannelId=8EFB008700000001/00000000548CFD8D, lastDispatchedFrom=10.112.210.1:55052, lastDispatchedTo=10.112.210.102:11210, requestId=124303, requestType=GetCollectionIdRequest, retried=4, retryReasons=[ENDPOINT_NOT_WRITABLE], service={bucket=default, collection=test, opaque=0x1e7b2, scope=_default, type=kv}, timeoutMs=10000, timings={dispatchMicros=2007922, serverMicros=0}}, cause=null}

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          There are no comments yet on this issue.

          People

            daschl Michael Nitschinger
            ashwin.govindarajulu Ashwin Govindarajulu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty