Uploaded image for project: 'Couchbase .NET client library'
  1. Couchbase .NET client library
  2. NCBC-645

[ssl] test Rb2Out-HYBRID stuck with 'Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException' in client logs

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Test Blocker
    • 2.0 Beta 2
    • 2.0 Beta
    • library
    • None

    Description

      c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
      ig.ini -I sdkd.args --variants HYBRID -d all:debug
      [3.21 INFO] (BRun run:436) Initializing S3
      [7.80 INFO] (BRun run:441) Initializing history database

      ============================================================
      Running Rb2Out-HYBRID: 2014-09-17 02:29:58
      rebalance/mode=out
      workload=HybridWorkloadGroup
      rebalance/count=2
      testcase=RebalanceScenario
      Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1208\Rb2Out-HYBRID\09-17-14\0341
      98\28852a8b5890fe94895277f14f7f4f66

      To re-run the test, copy/paste the following into the shell.
      You may also copy/paste (except the first line) into an argfile
      8<----------------------------------------
      ./stester \
      \ --rebalance-mode out --workload HybridWorkloadGroup
      \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec
      \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
      \ 8675 --cluster_node=10.3.121.134 --cluster_node=10.3.121.135
      \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206
      \ --cluster_ssh-username=root --cluster_ssh-password=couchbase
      --cluster_useSSL=True

      [447.89 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
      [448.91 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
      [449.93 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
      [449.94 INFO] (Scenario run:97) CHANGE phase done
      [449.95 INFO] (Scenario run:103) Starting REBOUND for 90 seconds
      [539.95 INFO] (Scenario run:105) REBOUND done. Will collect results
      [539.95 DEBUG] (Handle sendMessageAsync:183) > CANCEL@103.72
      [539.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Cancelling command 72 for Handle 103
      [539.97 DEBUG] (Handle receiveMessage:158) < CANCEL@103.72
      [539.97 DEBUG] (Handle sendMessageAsync:183) > CANCEL@104.72
      [539.98 INFO] (SDKD log:137) [Sdkd.Control|Info] Cancelling command 72 for Handle 104
      [2705.74 DEBUG] (Handle receiveMessage:158) < CANCEL@104.72
      [2706.66 DEBUG] (Handle sendMessageAsync:183) > CANCEL@105.72
      [2707.19 INFO] (SDKD log:137) [Sdkd.Control|Info] Cancelling command 72 for Handle 105
      Terminate batch job (Y/N)? Y

      part of the client logs when test stuck(02:37 - 03:14)

      2014-09-17 02:37:18,228 [7] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#870 on thread 7
      2014-09-17 02:37:18,227 [47] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: 40d8fbd1-7b5d-4334-a688-065d2575cca6 on 10.3.3.206:11207 - a17e5df5-7c63-4030-ba02-41309e68ea55
      2014-09-17 02:37:18,225 [21] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: abf3e988-cbfc-4795-8523-325094b9e719 on 10.3.121.134:11207 - 624b4df1-35dc-4842-8a23-7db2310b001b
      2014-09-17 02:37:18,232 [50] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#808 on thread 50
      2014-09-17 02:37:18,230 [9] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#800 on thread 9
      2014-09-17 02:37:18,231 [6] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#820 on thread 6
      2014-09-17 02:37:18,233 [43] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#828 on thread 43
      2014-09-17 02:37:18,253 [18] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#840 on thread 18
      2014-09-17 02:37:18,252 [28] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#800 on thread 28
      2014-09-17 02:37:18,254 [53] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#798 on thread 53
      2014-09-17 02:37:18,258 [42] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#870 on thread 42
      2014-09-17 02:37:18,258 [46] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 981 for key SimpleKeyREP5537REP5537REP5537REP5537REP5537 - rev798
      2014-09-17 02:37:18,210 [49] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 515 for key SimpleKeyREP5382REP5382REP5382REP5382REP5382 - rev870
      2014-09-17 02:37:19,297 [59] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.135:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 02:37:19,298 [13] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.136:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 02:37:19,299 [62] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.135:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 02:37:19,297 [20] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.135:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 02:37:19,299 [12] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.135:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 02:37:19,297 [15] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.135:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 02:37:19,301 [61] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.136:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 02:37:19,298 [60] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.136:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 02:37:19,300 [11] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.136:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 03:14:51,391 [7] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 427 for key SimpleKeyREP5352REP5352REP5352REP5352REP5352 - rev870
      2014-09-17 03:14:51,391 [30] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 226 for key SimpleKeyREP5697REP5697REP5697REP5697REP5697 - rev870
      2014-09-17 03:14:51,391 [24] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 958 for key SimpleKeyREP5964REP5964REP5964REP5964REP5964 - rev817
      2014-09-17 03:14:51,391 [48] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 456 for key SimpleKeyREP5799REP5799REP5799REP5799REP5799 - rev810
      2014-09-17 03:14:51,391 [8] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 461 for key SimpleKeyREP6028REP6028REP6028REP6028REP6028 - rev799
      2014-09-17 03:14:51,334 [49] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5382REP5382REP5382REP5382REP5382 using server 10.3.121.134:11207
      2014-09-17 03:14:51,391 [44] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 246 for key SimpleKeyREP5854REP5854REP5854REP5854REP5854 - rev870
      2014-09-17 03:14:51,391 [6] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 34 for key SimpleKeyREP5909REP5909REP5909REP5909REP5909 - rev820
      2014-09-17 03:14:51,391 [9] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 728 for key SimpleKeyREP5752REP5752REP5752REP5752REP5752 - rev800
      2014-09-17 03:14:51,391 [43] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 554 for key SimpleKeyREP5750REP5750REP5750REP5750REP5750 - rev828
      2014-09-17 03:14:51,391 [50] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 109 for key SimpleKeyREP5503REP5503REP5503REP5503REP5503 - rev808
      2014-09-17 03:14:51,391 [18] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 633 for key SimpleKeyREP5670REP5670REP5670REP5670REP5670 - rev840
      2014-09-17 03:14:51,391 [28] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 825 for key SimpleKeyREP5706REP5706REP5706REP5706REP5706 - rev800
      2014-09-17 03:14:51,391 [53] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 964 for key SimpleKeyREP5289REP5289REP5289REP5289REP5289 - rev798
      2014-09-17 03:14:51,391 [42] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 610 for key SimpleKeyREP5850REP5850REP5850REP5850REP5850 - rev870
      2014-09-17 03:14:51,334 [46] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5537REP5537REP5537REP5537REP5537 using server 10.3.3.206:11207
      2014-09-17 03:14:51,631 [45] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation succeeded 0 for key SimpleKeyREP5291REP5291REP5291REP5291REP5291
      2014-09-17 03:14:51,631 [47] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation succeeded 0 for key SimpleKeyREP5660REP5660REP5660REP5660REP5660
      2014-09-17 03:14:51,631 [21] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation succeeded 0 for key SimpleKeyREP5520REP5520REP5520REP5520REP5520
      2014-09-17 03:14:51,632 [52] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation succeeded 0 for key SimpleKeyREP5089REP5089REP5089REP5089REP5089
      2014-09-17 02:37:19,300 [63] ERROR Couchbase.Views.ViewClient - System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.121.136:18092
      at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
      at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
      — End of inner exception stack trace —
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
      — End of inner exception stack trace —
      2014-09-17 03:14:54,754 [7] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5352REP5352REP5352REP5352REP5352 using server 10.3.3.206:11207
      2014-09-17 03:14:54,756 [49] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire existing: c1a7c0f9-9ae6-406a-870f-221d1cd5abc7 | 10.3.121.134:11207 | [0, 2] - 45e4bde8-afd4-4084-b4fe-1952631c5f47 - Disposed: False
      2014-09-17 03:14:54,760 [18] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5670REP5670REP5670REP5670REP5670 using server 10.3.3.206:11207
      2014-09-17 03:14:54,760 [28] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5706REP5706REP5706REP5706REP5706 using server 10.3.3.206:11207
      2014-09-17 03:14:54,762 [46] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire existing: dfdafeeb-7f31-4104-94ea-a7c9c951aa6e | 10.3.3.206:11207 | [0, 2] - 728e6fe1-8778-4acd-b15d-832d832b0a1a - Disposed: False
      2014-09-17 03:14:54,760 [53] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5289REP5289REP5289REP5289REP5289 using server 10.3.3.206:11207
      2014-09-17 03:14:54,761 [42] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5850REP5850REP5850REP5850REP5850 using server 10.3.3.206:11207
      2014-09-17 03:14:54,749 [20] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - trying again: 1
      2014-09-17 03:14:54,750 [15] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - trying again: 1
      2014-09-17 03:14:54,760 [59] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - trying again: 1
      2014-09-17 03:14:54,747 [13] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - trying again: 1
      2014-09-17 03:14:54,748 [62] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - trying again: 1
      2014-09-17 03:14:54,760 [50] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5503REP5503REP5503REP5503REP5503 using server 10.3.121.134:11207
      2014-09-17 03:14:54,750 [12] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - trying again: 1
      2014-09-17 03:14:54,758 [9] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5752REP5752REP5752REP5752REP5752 using server 10.3.3.206:11207
      2014-09-17 03:14:54,754 [30] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5697REP5697REP5697REP5697REP5697 using server 10.3.121.134:11207
      2014-09-17 03:14:54,752 [60] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - trying again: 1
      2014-09-17 03:14:54,753 [11] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - trying again: 1
      2014-09-17 03:14:54,755 [48] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP5799REP5799REP5799REP5799REP5799 using server 10.3.3.206:11207

      Attachments

        1. NCBC-645.zip
          10.24 MB
        2. test_logs.txt
          70 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            jmorris Jeff Morris
            andreibaranouski Andrei Baranouski
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty