Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11244

[Windows] : Simple test broken : failing with error "Prepare join failed. Timeout connecting on port 8091"

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Test Blocker
    • 3.0
    • 3.0
    • ns_server
    • Security Level: Public
    • None
    • 3.0.0-750-rel
    • Triaged
    • Windows 64-bit
    • Yes

    Description

      Tests to Reproduce:
      ./testrunner -i /tmp/nodes-win-64.ini get-cbcollect-info=True -t xdcr.uniXDCR.unidirectional.load_with_ops,replicas=1,items=10000,value_size=128,ctopology=chain,rdirection=unidirection,doc-ops=update-delete

      Jenkins Ref Link:
      http://qa.hq.northscale.net/job/win_2008_x64--01_00--qe-sanity-P0/16/consoleFull
      Logs:
      [user:info,2014-05-29T1:57:38.911,ns_1@10.3.2.239:<0.901.0>:ns_log:crash_consumption_loop:70]Port server xdcr_proxy on node 'babysitter_of_ns_1@127.0.0.1' exited with status 3. Restarting. Messages: [

      {ns_ssl_proxy,stop,[],[]},
      {child_erlang,child_loop,2,[{file,"src/child_erlang.erl"},{line,121}]},
      {child_erlang,child_start,1,[{file,"src/child_erlang.erl"},{line,78}]
      },
      {init,start_it,1,[{file,"init.erl"},{line,1055}]},
      {init,start_em,1,[{file,"init.erl"},{line,1034}]
      }]
      [user:debug,2014-05-29T1:57:38.927,ns_1@10.3.2.239:<0.901.0>:ns_log:crash_consumption_loop:70]Port server moxi on node 'babysitter_of_ns_1@127.0.0.1' exited with status 0. Restarting. Messages: WARNING: curl error: Failed to connect to 127.0.0.1 port 8091: Connection refused from: http://127.0.0.1:8091/pools/default/saslBucketsStreaming
      ERROR: could not contact REST server(s): http://127.0.0.1:8091/pools/default/saslBucketsStreaming
      WARNING: curl error: Failed to connect to 127.0.0.1 port 8091: Connection refused from: http://127.0.0.1:8091/pools/default/saslBucketsStreaming
      ERROR: could not contact REST server(s): http://127.0.0.1:8091/pools/default/saslBucketsStreaming
      EOL on stdin. Exiting
      [ns_server:debug,2014-05-29T1:57:39.958,ns_1@10.3.2.239:<0.1090.0>:ns_pubsub:do_subscribe_link:136]Parent process of subscription {ns_config_events,<0.1088.0>} exited with reason {badmatch,
      {is_pid,
      false,
      {badrpc,
      {'EXIT',
      {{badmatch,
      {error,
      {already_started, <13913.92.0>}}},
      [{ns_child_ports_sup,
      launch_port,
      1,
      [{file, "src/ns_child_ports_sup.erl"},
      {line, 96}]},
      {lists,
      foreach,
      2,
      [{file, "lists.erl"},
      {line, 1323}]},
      {ns_child_ports_sup,
      set_dynamic_children,
      1,
      [{file, "src/ns_child_ports_sup.erl"},
      {line, 69}]},
      {rpc,
      'handle_call_call/6-fun-0',
      5,
      [{file, "rpc.erl"},
      {line, 205}]}]}}}}}
      [error_logger:error,2014-05-29T1:57:39.958,ns_1@10.3.2.239:error_logger<0.6.0>:ale_error_logger_handler:do_log:207]
      =========================CRASH REPORT=========================
      crasher:
      initial call: ns_ports_setup:setup_body_tramp/0
      pid: <0.1088.0>
      registered_name: ns_ports_setup
      exception error: no match of right hand side value
      {is_pid,false,
      {badrpc,
      {'EXIT',
      {{badmatch,
      {error,{already_started,<13913.92.0>}}},
      [{ns_child_ports_sup,launch_port,1,
      [{file,"src/ns_child_ports_sup.erl"},
      {line,96}]},
      {lists,foreach,2,
      [{file,"lists.erl"},{line,1323}]},
      {ns_child_ports_sup,set_dynamic_children,
      1,
      [{file,"src/ns_child_ports_sup.erl"},
      {line,69}]},
      {rpc,'handle_call_call/6-fun-0',5,
      [{file,"rpc.erl"},{line,205}]}]}}}}
      in function ns_ports_setup:set_childs_and_loop/1 (src/ns_ports_setup.erl, line 58)
      in call from misc:delaying_crash/2 (src/misc.erl, line 1546)
      ancestors: [ns_server_sup,ns_server_cluster_sup,<0.56.0>]
      messages: [check_childs_update,check_childs_update,check_childs_update,
      check_childs_update,check_childs_update,check_childs_update]
      links: [<0.896.0>,<0.1090.0>]
      dictionary: []
      trap_exit: false
      status: running
      heap_size: 196650
      stack_size: 27
      reductions: 23593
      neighbours:

      [ns_server:debug,2014-05-29T1:57:54.958,ns_1@10.3.2.239:upr_consumer_conn-default-ns_1@10.3.3.38<0.1777.0>:upr_proxy:terminate:78]Terminating. Disconnecting from socket #Port<0.12548>
      [error_logger:error,2014-05-29T1:57:54.958,ns_1@10.3.2.239:error_logger<0.6.0>:ale_error_logger_handler:do_log:207]** Generic server <0.1777.0> terminating
      ** Last message in was {tcp,#Port<0.12548>, <<129,85,0,0,0,0,0,0,0,0,0,0,12,0,0,0,0,0,0,0,0,0, 0,0>>}
      ** When Server state == {state,#Port<0.12548>,
      {consumer, "replication:ns_1@10.3.3.38->ns_1@10.3.2.239:default", 'ns_1@10.3.2.239',"default"},
      <<>>,upr_consumer_conn,
      {state,idle,[]},
      #Port<0.12550>,<0.1779.0>}
      ** Reason for termination ==
      ** badmatch,{error,closed,
      [{upr_proxy,handle_packet,2,[{file,"src/upr_proxy.erl"},{line,130}]},
      {mc_replication,process_data,3,
      [{file,"src/mc_replication.erl"},{line,59}]},
      {mc_replication,process_data,4,
      [{file,"src/mc_replication.erl"},{line,76}]
      },
      {upr_proxy,handle_info,2,[{file,"src/upr_proxy.erl"},{line,84}]},
      {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,604}]},
      {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}

      [error_logger:error,2014-05-29T1:57:54.958,ns_1@10.3.2.239:error_logger<0.6.0>:ale_error_logger_handler:do_log:207]
      =========================CRASH REPORT=========================
      crasher:
      initial call: upr_proxy:init/1
      pid: <0.1777.0>
      registered_name: 'upr_consumer_conn-default-ns_1@10.3.3.38'
      exception exit: badmatch,{error,closed,
      [{upr_proxy,handle_packet,2,
      [{file,"src/upr_proxy.erl"},{line,130}]},
      {mc_replication,process_data,3,
      [{file,"src/mc_replication.erl"},
      {line,59}]},
      {mc_replication,process_data,4,
      [{file,"src/mc_replication.erl"},
      {line,76}]},
      {upr_proxy,handle_info,2,
      [{file,"src/upr_proxy.erl"},{line,84}]
      },
      {gen_server,handle_msg,5,
      [{file,"gen_server.erl"},{line,604}]},
      {proc_lib,init_p_do_apply,3,
      [{file,"proc_lib.erl"},{line,239}]}]}
      in function gen_server:terminate/6 (gen_server.erl, line 744)
      ancestors: ['upr_replicator-default-ns_1@10.3.3.38','upr_sup-default',
      'single_bucket_sup-default',<0.1150.0>]
      messages: [{'EXIT',<0.1776.0>,shutdown}]
      links: [<0.1776.0>]
      dictionary: []
      trap_exit: true
      status: running
      heap_size: 2586
      stack_size: 27
      reductions: 243693
      neighbours:
      [user:debug,2014-05-29T1:59:44.240,ns_1@10.3.2.239:<0.901.0>:ns_log:crash_consumption_loop:70]Port server memcached on node 'babysitter_of_ns_1@127.0.0.1' exited with status 0. Restarting. Messages: Thu May 29 01:59:43.126953 Pacific Daylight Time 3: (default) UPR (Producer) eq_uprq:replication:ns_1@10.3.2.239->ns_1@10.3.2.243:default - (vb 84) Stream closing, 0 items sent from disk, 0 items sent from memory, 55 was last seqno sent
      Thu May 29 01:59:43.126953 Pacific Daylight Time 3: (default) UPR (Producer) eq_uprq:replication:ns_1@10.3.2.239->ns_1@10.3.2.243:default - (vb 85) Stream closing, 0 items sent from disk, 0 items sent from memory, 56 was last seqno sent
      Thu May 29 01:59:43.155273 Pacific Daylight Time 3: (default) Shutting down tap connections!
      Thu May 29 01:59:43.155273 Pacific Daylight Time 3: (default) Shutting down upr connections!
      'die!' on stdin. Exiting super-quickly

      [user:info,2014-05-29T2:00:00.880,ns_1@10.3.2.239:<0.9767.0>:ns_log:crash_consumption_loop:70]Port server xdcr_proxy on node 'babysitter_of_ns_1@127.0.0.1' exited with status 3. Restarting. Messages: [{ns_ssl_proxy,stop,[],[]}

      ,
      {child_erlang,child_loop,2,[

      {file,"src/child_erlang.erl"},{line,121}]},
      {child_erlang,child_start,1,[{file,"src/child_erlang.erl"}

      ,

      {line,78}

      ]},
      {init,start_it,1,[

      {file,"init.erl"},{line,1055}]},
      {init,start_em,1,[{file,"init.erl"}

      ,

      {line,1034}

      ]}]
      [user:debug,2014-05-29T2:00:00.896,ns_1@10.3.2.239:<0.9767.0>:ns_log:crash_consumption_loop:70]Port server moxi on node 'babysitter_of_ns_1@127.0.0.1' exited with status 0. Restarting. Messages: ERROR: could not contact REST server(s): http://127.0.0.1:8091/pools/default/saslBucketsStreaming
      WARNING: curl error: Failed to connect to 127.0.0.1 port 8091: Connection refused from: http://127.0.0.1:8091/pools/default/saslBucketsStreaming
      ERROR: could not contact REST server(s): http://127.0.0.1:8091/pools/default/saslBucketsStreaming
      WARNING: curl error: Failed to connect to 127.0.0.1 port 8091: Connection refused from: http://127.0.0.1:8091/pools/default/saslBucketsStreaming
      EOL on stdin. Exiting
      [ns_server:debug,2014-05-29T2:00:01.458,ns_1@10.3.2.239:ns_heart_slow_status_updater<0.9801.0>:ns_heart:current_status_slow_inner:259]Ignoring failure to grab system stats:

      {error,no_samples}

      Uploading Logs.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Meenakshi Meenakshi Goel
            Meenakshi Meenakshi Goel
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty