Details
-
Bug
-
Resolution: Fixed
-
Major
-
None
-
None
-
None
Description
client version : couchbase-net-client 2.4.2
server version : 4.5.1 GA
Reproduction steps:
- run 2 or more cluster
- open bucket using one of the node host (entry point)
- keep upsert.
- while upsert, failover the entry point and leave it for more than 15 seconds.
Expected : upsert should work with active nodes
Current : upsert hangs
Details : net client waits for lock which is not released by previous call to
https://github.com/couchbase/couchbase-net-client/blob/master/Src/Couchbase/Core/Server.cs#L414
Here are two call stacks happened exactly 15 second after the other lock
First lock right after node is failed over:
> Couchbase.NetClient.dll!Couchbase.Core.Server.CheckOnline(bool isDead) Line 416 C#
Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.MultiplexingConnection>.Release(Couchbase.IO.MultiplexingConnection connection) Line 214 C#
Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.MultiplexingConnection>.Couchbase.IO.IConnectionPool.Release(Couchbase.IO.IConnection connection) Line 315 C#
Couchbase.NetClient.dll!Couchbase.IO.Services.MultiplexingIOService.CheckConnection() Line 378 C#
Couchbase.NetClient.dll!Couchbase.IO.Services.MultiplexingIOService.ExecuteAsync<object>(Couchbase.IO.Operations.IOperation<object> operation, Couchbase.IO.IConnection connection) Line 197 C#
Couchbase.NetClient.dll!Couchbase.IO.Services.MultiplexingIOService.ExecuteAsync<object>(Couchbase.IO.Operations.IOperation<object> operation) Line 209 C#
Couchbase.NetClient.dll!Couchbase.Core.Server.SendAsync<object>(Couchbase.IO.Operations.IOperation<object> operation) Line 632 C#
Couchbase.NetClient.dll!Couchbase.Core.Buckets.CouchbaseRequestExecuter.SendWithRetryAsync<object>(Couchbase.IO.Operations.IOperation<object> operation, System.Threading.Tasks.TaskCompletionSource<Couchbase.IOperationResult<object>> tcs, System.Threading.CancellationTokenSource cts) Line 740 C#
Couchbase.NetClient.dll!Couchbase.Core.Buckets.CouchbaseRequestExecuter.SendWithDurabilityAsync<object>(Couchbase.IO.Operations.IOperation<object> operation, bool deletion, Couchbase.ReplicateTo replicateTo, Couchbase.PersistTo persistTo) Line 345 C#
Couchbase.NetClient.dll!Couchbase.CouchbaseBucket.UpsertAsync<object>(string key, object value, ulong cas, uint expiration, Couchbase.ReplicateTo replicateTo, Couchbase.PersistTo persistTo) Line 3412 C#
Couchbase.NetClient.dll!Couchbase.CouchbaseBucket.UpsertAsync<object>(string key, object value, Couchbase.ReplicateTo replicateTo, Couchbase.PersistTo persistTo) Line 3362 C#
Sdkd.dll!Sdkd.Commands.SetCommand.ExecuteSingleAsync(System.Collections.Generic.Dictionary<string, string> batch) Line 67 C#
Sdkd.dll!Sdkd.Commands.KVCommand.Run(int handleId) Line 42 C#
SdkdConsole.exe!SdkdConsole.Daemon.Handle.DispatchCommand(Sdkd.Protocol.Request req, Couchbase.Core.IBucket cli) Line 271 C#
SdkdConsole.exe!SdkdConsole.Daemon.Handle.Run() Line 141 C#
Second lock after 15 seconds :
Couchbase.NetClient.dll!Couchbase.Core.Server.CheckOnline(bool isDead) Line 416 C#
Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.MultiplexingConnection>.Release(Couchbase.IO.MultiplexingConnection connection) Line 214 C#
Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.MultiplexingConnection>.Couchbase.IO.IConnectionPool.Release(Couchbase.IO.IConnection connection) Line 315 C#
Couchbase.NetClient.dll!Couchbase.IO.Services.MultiplexingIOService.CheckConnection() Line 393 C#
Couchbase.NetClient.dll!Couchbase.IO.Services.MultiplexingIOService.ExecuteAsync<object>(Couchbase.IO.Operations.IOperation<object> operation, Couchbase.IO.IConnection connection) Line 197 C#
Couchbase.NetClient.dll!Couchbase.IO.Services.MultiplexingIOService.ExecuteAsync<object>(Couchbase.IO.Operations.IOperation<object> operation) Line 209 C#
Couchbase.NetClient.dll!Couchbase.Core.Server.SendAsync<object>(Couchbase.IO.Operations.IOperation<object> operation) Line 632 C#
Couchbase.NetClient.dll!Couchbase.Core.Buckets.CouchbaseRequestExecuter.SendWithRetryAsync<object>(Couchbase.IO.Operations.IOperation<object> operation, System.Threading.Tasks.TaskCompletionSource<Couchbase.IOperationResult<object>> tcs, System.Threading.CancellationTokenSource cts) Line 740 C#
Couchbase.NetClient.dll!Couchbase.Core.Buckets.CouchbaseRequestExecuter.SendWithDurabilityAsync<object>(Couchbase.IO.Operations.IOperation<object> operation, bool deletion, Couchbase.ReplicateTo replicateTo, Couchbase.PersistTo persistTo) Line 345 C#
Couchbase.NetClient.dll!Couchbase.CouchbaseBucket.UpsertAsync<object>(string key, object value, ulong cas, uint expiration, Couchbase.ReplicateTo replicateTo, Couchbase.PersistTo persistTo) Line 3412 C#
Couchbase.NetClient.dll!Couchbase.CouchbaseBucket.UpsertAsync<object>(string key, object value, Couchbase.ReplicateTo replicateTo, Couchbase.PersistTo persistTo) Line 3362 C#
Sdkd.dll!Sdkd.Commands.SetCommand.ExecuteSingleAsync(System.Collections.Generic.Dictionary<string, string> batch) Line 67 C#
Sdkd.dll!Sdkd.Commands.KVCommand.Run(int handleId) Line 42 C#
SdkdConsole.exe!SdkdConsole.Daemon.Handle.DispatchCommand(Sdkd.Protocol.Request req, Couchbase.Core.IBucket cli) Line 271 C#
SdkdConsole.exe!SdkdConsole.Daemon.Handle.Run() Line 141 C#
Attachments
Issue Links
- relates to
-
NCBC-1394 OperationTimeout and high memory consumption when swapping in a node
- Resolved