Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6664

[windows 7] bucket creation on windows 7 sometimes timeout if bucket is created immediately after another bucket was deleted

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 2.0
    • Component/s: couchbase-bucket
    • Security Level: Public
    • Environment:
      windows 7 64-bit
      build 1723
      single node

      Description

      unable to create a bucket with replica=2
      [error_logger:error,2012-09-13T16:31:17.647,ns_1@10.3.2.188:error_logger:ale_error_logger_handler:log_report:72]
      =========================CRASH REPORT=========================
      crasher:
      initial call: ns_memcached:init/1
      pid: <0.27705.2>
      registered_name: []
      exception exit: {ensure_bucket_failed,{error,function_clause}}
      in function gen_server:init_it/6
      ancestors: ['single_bucket_sup-2replica_90e83784-3382-4a59-8e49-586975bceb76',
      <0.27670.2>]
      messages: []
      links: <0.64.0>,<0.27671.2>,#Port<0.47362>
      dictionary: []
      trap_exit: true
      status: running
      heap_size: 28657
      stack_size: 24
      reductions: 3751
      neighbours:

      1. win7debuglogs.tar.gz
        8.06 MB
        Iryna
      2. win7single.tar.gz
        403 kB
        Iryna
      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        I need full diags. Or at least all debug.* logs

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - I need full diags. Or at least all debug.* logs
        Hide
        farshid Farshid Ghods (Inactive) added a comment -

        Iryna,

        please use the fixVersion as 2.0 beta or 2.0 and also set the priorities accordingly/
        being unable to create a bukcet on windows 7 is a blocker bug

        Show
        farshid Farshid Ghods (Inactive) added a comment - Iryna, please use the fixVersion as 2.0 beta or 2.0 and also set the priorities accordingly/ being unable to create a bukcet on windows 7 is a blocker bug
        Hide
        iryna iryna added a comment -

        all debug logs attached

        Show
        iryna iryna added a comment - all debug logs attached
        Hide
        iryna iryna added a comment -

        logs attached

        Show
        iryna iryna added a comment - logs attached
        Hide
        iryna iryna added a comment -

        ip of machine is 10.3.2.188
        Jin, can you take a look?

        Show
        iryna iryna added a comment - ip of machine is 10.3.2.188 Jin, can you take a look?
        Hide
        jin Jin Lim added a comment -

        1) NS Server reported TIMEOUT on deleting the bucket, "2replica_90e83784-3382-4a59-8e49-586975bceb76"

        [ns_server:warn,2012-09-13T16:31:47.616,ns_1@10.3.2.188:<0.430.0>:ns_orchestrator:idle:418]Nodes ['ns_1@10.3.2.188'] failed to delete bucket "2replica_90e83784-3382-4a59-8e49-586975bceb76" within expected time.

        2) Prior to the TIMEOUT there were any error reported on mecached(ep_engine) side. However, there were this repeating CRASH from ns_server side.

        [error_logger:error,2012-09-13T16:31:17.678,ns_1@10.3.2.188:error_logger:ale_error_logger_handler:log_msg:76]Error in process <0.27670.2> on node 'ns_1@10.3.2.188' with exit value: badmatch,{error,shutdown,[

        {single_bucket_sup,'-start_link/1-fun-0-',2}

        ]}

        [ns_server:debug,2012-09-13T16:31:18.553,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop:111]Exception from query_vbucket_states of "2replica_90e83784-3382-4a59-8e49-586975bceb76":'ns_1@10.3.2.188'
        {'EXIT',
        {noproc,
        {gen_server,call,
        [

        {'janitor_agent-2replica_90e83784-3382-4a59-8e49-586975bceb76', 'ns_1@10.3.2.188'},
        query_vbucket_states,infinity]}}}
        [ns_server:debug,2012-09-13T16:31:18.553,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop_next_step:116]Waiting for "2replica_90e83784-3382-4a59-8e49-586975bceb76" on 'ns_1@10.3.2.188'
        [ns_server:debug,2012-09-13T16:31:19.569,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop:111]Exception from query_vbucket_states of "2replica_90e83784-3382-4a59-8e49-586975bceb76":'ns_1@10.3.2.188'
        {'EXIT',
        {noproc,
        {gen_server,call,
        [{'janitor_agent-2replica_90e83784-3382-4a59-8e49-586975bceb76', 'ns_1@10.3.2.188'}

        ,
        query_vbucket_states,infinity]}}}
        [ns_server:debug,2012-09-13T16:31:19.569,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop_next_step:116]Waiting for "2replica_90e83784-3382-4a59-8e49-586975bceb76" on 'ns_1@10.3.2.188'
        [ns_server:debug,2012-09-13T16:31:20.585,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop:111]Exception from query_vbucket_states of "2replica_90e83784-3382-4a59-8e49-586975bceb76":'ns_1@10.3.2.188'
        {'EXIT',
        {noproc,
        {gen_server,call,
        [

        {'janitor_agent-2replica_90e83784-3382-4a59-8e49-586975bceb76', 'ns_1@10.3.2.188'},
        query_vbucket_states,infinity]}}}
        [ns_server:debug,2012-09-13T16:31:20.585,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop_next_step:116]Waiting for "2replica_90e83784-3382-4a59-8e49-586975bceb76" on 'ns_1@10.3.2.188'
        [ns_server:debug,2012-09-13T16:31:21.600,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop:111]Exception from query_vbucket_states of "2replica_90e83784-3382-4a59-8e49-586975bceb76":'ns_1@10.3.2.188'
        {'EXIT',
        {noproc,
        {gen_server,call,
        [{'janitor_agent-2replica_90e83784-3382-4a59-8e49-586975bceb76', 'ns_1@10.3.2.188'}

        ,
        query_vbucket_states,infinity]}}}
        [ns_server:debug,2012-09-13T16:31:21.600,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop_next_step:116]Waiting for "2replica_90e83784-3382-4a59-8e49-586975bceb76" on 'ns_1@10.3.2.188'
        [ns_doctor:debug,2012-09-13T16:31:38.788,ns_1@10.3.2.188:ns_doctor:ns_doctor:handle_info:136]Current node statuses:
        [{'ns_1@10.3.2.188',

        Can we rerun the test with increased memcached verbose level? wget O -q --user=Administrator --password=password --post-data='ns_config:update_key(

        {node, node(), memcached}

        , fun (PList) -> lists:keyreplace(verbosity, 1, PList,

        {verbosity, "-vvv"}

        ) end).' http://10.3.2.188:8091/diag/eval

        Show
        jin Jin Lim added a comment - 1) NS Server reported TIMEOUT on deleting the bucket, "2replica_90e83784-3382-4a59-8e49-586975bceb76" [ns_server:warn,2012-09-13T16:31:47.616,ns_1@10.3.2.188:<0.430.0>:ns_orchestrator:idle:418] Nodes ['ns_1@10.3.2.188'] failed to delete bucket "2replica_90e83784-3382-4a59-8e49-586975bceb76" within expected time. 2) Prior to the TIMEOUT there were any error reported on mecached(ep_engine) side. However, there were this repeating CRASH from ns_server side. [error_logger:error,2012-09-13T16:31:17.678,ns_1@10.3.2.188:error_logger:ale_error_logger_handler:log_msg:76] Error in process <0.27670.2> on node 'ns_1@10.3.2.188' with exit value: badmatch,{error,shutdown ,[ {single_bucket_sup,'-start_link/1-fun-0-',2} ]} [ns_server:debug,2012-09-13T16:31:18.553,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop:111] Exception from query_vbucket_states of "2replica_90e83784-3382-4a59-8e49-586975bceb76":'ns_1@10.3.2.188' {'EXIT', {noproc, {gen_server,call, [ {'janitor_agent-2replica_90e83784-3382-4a59-8e49-586975bceb76', 'ns_1@10.3.2.188'}, query_vbucket_states,infinity]}}} [ns_server:debug,2012-09-13T16:31:18.553,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop_next_step:116] Waiting for "2replica_90e83784-3382-4a59-8e49-586975bceb76" on 'ns_1@10.3.2.188' [ns_server:debug,2012-09-13T16:31:19.569,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop:111] Exception from query_vbucket_states of "2replica_90e83784-3382-4a59-8e49-586975bceb76":'ns_1@10.3.2.188' {'EXIT', {noproc, {gen_server,call, [{'janitor_agent-2replica_90e83784-3382-4a59-8e49-586975bceb76', 'ns_1@10.3.2.188'} , query_vbucket_states,infinity]}}} [ns_server:debug,2012-09-13T16:31:19.569,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop_next_step:116] Waiting for "2replica_90e83784-3382-4a59-8e49-586975bceb76" on 'ns_1@10.3.2.188' [ns_server:debug,2012-09-13T16:31:20.585,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop:111] Exception from query_vbucket_states of "2replica_90e83784-3382-4a59-8e49-586975bceb76":'ns_1@10.3.2.188' {'EXIT', {noproc, {gen_server,call, [ {'janitor_agent-2replica_90e83784-3382-4a59-8e49-586975bceb76', 'ns_1@10.3.2.188'}, query_vbucket_states,infinity]}}} [ns_server:debug,2012-09-13T16:31:20.585,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop_next_step:116] Waiting for "2replica_90e83784-3382-4a59-8e49-586975bceb76" on 'ns_1@10.3.2.188' [ns_server:debug,2012-09-13T16:31:21.600,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop:111] Exception from query_vbucket_states of "2replica_90e83784-3382-4a59-8e49-586975bceb76":'ns_1@10.3.2.188' {'EXIT', {noproc, {gen_server,call, [{'janitor_agent-2replica_90e83784-3382-4a59-8e49-586975bceb76', 'ns_1@10.3.2.188'} , query_vbucket_states,infinity]}}} [ns_server:debug,2012-09-13T16:31:21.600,ns_1@10.3.2.188:<0.27676.2>:janitor_agent:new_style_query_vbucket_states_loop_next_step:116] Waiting for "2replica_90e83784-3382-4a59-8e49-586975bceb76" on 'ns_1@10.3.2.188' [ns_doctor:debug,2012-09-13T16:31:38.788,ns_1@10.3.2.188:ns_doctor:ns_doctor:handle_info:136] Current node statuses: [{'ns_1@10.3.2.188', Can we rerun the test with increased memcached verbose level? wget O -q --user=Administrator --password=password --post-data='ns_config:update_key( {node, node(), memcached} , fun (PList) -> lists:keyreplace(verbosity, 1, PList, {verbosity, "-vvv"} ) end).' http://10.3.2.188:8091/diag/eval
        Hide
        chiyoung Chiyoung Seo added a comment -

        Assign it to Alk for the initial investigation.

        Show
        chiyoung Chiyoung Seo added a comment - Assign it to Alk for the initial investigation.
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Logs were rotated past incident in ticket's description

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Logs were rotated past incident in ticket's description
        Hide
        jin Jin Lim added a comment -

        Iryna & Alk,
        The log snippets from my initial investigation above are directly from the node (ssh to the node and peeked at the logs under ~/logs) Give either Alk the direct access info or collect entire logs for him. Thanks.

        Show
        jin Jin Lim added a comment - Iryna & Alk, The log snippets from my initial investigation above are directly from the node (ssh to the node and peeked at the logs under ~/logs) Give either Alk the direct access info or collect entire logs for him. Thanks.
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        It may be easier to reproduce and right after problem happens grab me diags.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - It may be easier to reproduce and right after problem happens grab me diags .

          People

          • Assignee:
            iryna iryna
            Reporter:
            iryna iryna
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes