Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4669

"reason":"{<<\"shutdown\ , [{'DOWN' and eventually reason":"req_timedout when running views during rebalance operation

    XMLWordPrintable

Details

    Description

      build 541 has the most recent fixes which was merged to epengine and couchdb

      cluster has 6 nodes - 200k items - under 1k ops/sec load

      rebalance hangs near completion and all view queries fail and eventually they all time out

      http://10.114.237.71:8092/default/_design/dev_players/_view/players?full_set=true&connection_timeout=60000

      took 1.04649209976 seconds
      {"rows":[

      {"key":null,"value":109441}

      ]}

      took 3.05865216255 seconds
      {"rows":[

      {"key":null,"value":102793}

      ],
      "errors":[
      {"from":"http://10.202.101.172:8092/_view_merge/","reason":"{<<\"shutdown\">>,\n <<\"{gen_server,call,[<0.16751.3>,

      {pread_iolist,41058}

      ,infinity]}\">>}"}
      ]}

      took 120.888384104 seconds
      {"rows":[

      {"key":null,"value":104536}

      ],
      "errors":[
      {"from":"http://10.202.101.172:8092/_view_merge/","reason":"{<<\"{function_clause,
      n [{couch_set_view,handle_info,
      n [{'DOWN',#Ref<0.0.116.88493>,process,<0.25834.3>,
      n {badmatch,
      n {error,
      n {badmatch,
      n {error,
      n badmatch,
      n {file_already_opened,
      n \\\"/mnt/default/66.couch\\\"
      ,
      n [

      {couch_db_set,'-init/1-fun-0-',4}

      ,
      n

      {lists,foldl,3}

      ,
      n

      {couch_db_set,init,1}

      ,
      n

      {gen_server,init_it,6}

      ,
      n

      {proc_lib,init_p_do_apply,3}]}}}}}},
      n {server,\\\"/mnt\\\",<0.16314.3>}]},
      n {gen_server,handle_msg,5},
      n {proc_lib,init_p_do_apply,3}

      ]}\">>,\n <<\"{gen_server,call,
      n [couch_set_view,
      n {get_group_server,<<\\\"default\\\">>,
      n {set_view_group,
      n <<126,233,23,3,126,62,146,158,97,21,49,212,152,252,155,124>>,
      n nil,<<\\\"default\\\">>,<<\\\"_design/dev_players\\\">>,<<\\\"javascript\\\">>,[],
      n [{set_view,0,[],[],[],
      n <<\\\"function (doc) {\\\\n if(doc.email)

      {\\\\n emit(doc._id, doc.city);\\\\n }

      \\\\n}\\\">>,
      n nil,
      n [

      {<<\\\"players\\\">>,<<\\\"_count\\\">>}

      ],
      n []}],
      n

      {[]}

      ,
      n nil,nil,nil,nil,nil,nil,undefined,nil,nil}},
      n infinity]}\">>}"}
      ]}

      took 0.724260807037 seconds
      {"rows":[

      {"key":null,"value":104731}

      ],
      "errors":[

      {"from":"http://10.202.101.172:8092/_view_merge/","reason":"req_timedout"}

      ]}

      took 60.014398098 seconds
      {"rows":[

      {"key":null,"value":133065}

      ],
      "errors":[

      {"from":"http://10.202.101.172:8092/_view_merge/","reason":"req_timedout"}

      ]}

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Aliaksey Artamonau Aliaksey Artamonau (Inactive)
            farshid Farshid Ghods (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty