Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-61689

Closing CB's cluster in COPY TO KV can be very spammy

    XMLWordPrintable

Details

    • Task
    • Resolution: Fixed
    • Major
    • Goldfish GA
    • Goldfish GA
    • analytics
    • 0
    • Analytics Sprint 41, Analytics Sprint 42

    Description

      After validation/execution is done, the couchbase cluster instance is closed/shutdown through our CleanupUtils, however, this operation seems to always fail, and so we log a warning about it. This can get very spammy and fill up the logs very fast as this seems to always happen.

      2024-04-29T08:46:14.722+00:00 WARN CBAS.util.CleanupUtils [SAO:JID:0.38:TAID:TID:ANID:ODID:3:0:13:0] Failure closing a closeable resource com.couchbase.client.java.env.ClusterEnvironmentreactor.core.Exceptions$ReactiveException: java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 10000ms in 'ignoreThen' (and no fallback has been configured)    at reactor.core.Exceptions.propagate(Exceptions.java:408) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:101) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.Mono.block(Mono.java:1712) ~[reactor-core-3.5.8.jar:3.5.8]    at com.couchbase.client.core.env.CoreEnvironment.shutdown(CoreEnvironment.java:550) ~[core-io-2.5.2.jar:?]    at com.couchbase.client.core.env.CoreEnvironment.shutdown(CoreEnvironment.java:560) ~[core-io-2.5.2.jar:?]    at com.couchbase.client.core.env.CoreEnvironment.close(CoreEnvironment.java:612) ~[core-io-2.5.2.jar:?]    at org.apache.hyracks.api.util.CleanupUtils.close(CleanupUtils.java:137) ~[hyracks-api-7.6.101-1345.jar:7.6.101-1345]    at com.couchbase.analytics.writer.KVExternalWriter.close(KVExternalWriter.java:211) ~[cbas-connector-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.algebricks.runtime.operators.writer.SinkExternalWriterRuntime.close(SinkExternalWriterRuntime.java:83) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.api.util.CleanupUtils.close(CleanupUtils.java:66) ~[hyracks-api-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.close(AbstractOneInputOneOutputOneFramePushRuntime.java:65) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$1.close(AlgebricksMetaOperatorDescriptor.java:233) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.api.util.CleanupUtils.close(CleanupUtils.java:66) ~[hyracks-api-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.close(IndexSearchOperatorNodePushable.java:332) ~[hyracks-storage-am-common-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.algebricks.runtime.operators.std.EmptyTupleSourceRuntimeFactory$1.close(EmptyTupleSourceRuntimeFactory.java:61) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$SourcePushRuntime.initialize(AlgebricksMetaOperatorDescriptor.java:181) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$0(SuperActivityOperatorNodePushable.java:233) ~[hyracks-api-7.6.101-1345.jar:7.6.101-1345]    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]    at java.base/java.lang.Thread.run(Thread.java:840) [?:?]    Suppressed: java.lang.Exception: #block terminated with an error        at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:103) ~[reactor-core-3.5.8.jar:3.5.8]        at reactor.core.publisher.Mono.block(Mono.java:1712) ~[reactor-core-3.5.8.jar:3.5.8]        at com.couchbase.client.core.env.CoreEnvironment.shutdown(CoreEnvironment.java:550) ~[core-io-2.5.2.jar:?]        at com.couchbase.client.core.env.CoreEnvironment.shutdown(CoreEnvironment.java:560) ~[core-io-2.5.2.jar:?]        at com.couchbase.client.core.env.CoreEnvironment.close(CoreEnvironment.java:612) ~[core-io-2.5.2.jar:?]        at org.apache.hyracks.api.util.CleanupUtils.close(CleanupUtils.java:137) ~[hyracks-api-7.6.101-1345.jar:7.6.101-1345]        at com.couchbase.analytics.writer.KVExternalWriter.close(KVExternalWriter.java:211) ~[cbas-connector-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.algebricks.runtime.operators.writer.SinkExternalWriterRuntime.close(SinkExternalWriterRuntime.java:83) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.api.util.CleanupUtils.close(CleanupUtils.java:66) ~[hyracks-api-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.close(AbstractOneInputOneOutputOneFramePushRuntime.java:65) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$1.close(AlgebricksMetaOperatorDescriptor.java:233) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.api.util.CleanupUtils.close(CleanupUtils.java:66) ~[hyracks-api-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.close(IndexSearchOperatorNodePushable.java:332) ~[hyracks-storage-am-common-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.algebricks.runtime.operators.std.EmptyTupleSourceRuntimeFactory$1.close(EmptyTupleSourceRuntimeFactory.java:61) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$SourcePushRuntime.initialize(AlgebricksMetaOperatorDescriptor.java:181) ~[algebricks-runtime-7.6.101-1345.jar:7.6.101-1345]        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$0(SuperActivityOperatorNodePushable.java:233) ~[hyracks-api-7.6.101-1345.jar:7.6.101-1345]        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]        at java.base/java.lang.Thread.run(Thread.java:840) [?:?]Caused by: java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 10000ms in 'ignoreThen' (and no fallback has been configured)    at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.handleTimeout(FluxTimeout.java:295) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.doTimeout(FluxTimeout.java:280) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.FluxTimeout$TimeoutTimeoutSubscriber.onNext(FluxTimeout.java:419) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.FluxOnErrorReturn$ReturnSubscriber.onError(FluxOnErrorReturn.java:199) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.MonoDelay.subscribe(MonoDelay.java:64) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.Mono.subscribe(Mono.java:4495) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.onSubscribe(FluxTimeout.java:155) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:50) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.Mono.subscribe(Mono.java:4495) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.Mono.subscribe(Mono.java:4495) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.MonoIgnorePublisher.subscribe(MonoIgnorePublisher.java:57) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.Mono.subscribe(Mono.java:4495) ~[reactor-core-3.5.8.jar:3.5.8]    at reactor.core.publisher.Mono.block(Mono.java:1711) ~[reactor-core-3.5.8.jar:3.5.8]    ... 18 more

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Hussain.Towaileb Hussain Towaileb
            Hussain.Towaileb Hussain Towaileb
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty