Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0-developer-preview-4
    • Fix Version/s: 2.0-developer-preview-4
    • Component/s: None
    • Security Level: Public
    • Labels:
      None

      Description

      Created 10 nodes cluster. Created a view: {"reduce":{"map":"function (doc)

      {\n emit(doc._id, null);\n}

      ","reduce":"_count"}. Uploaded 100k json documents using mcsoda. Queried a view with stale=false. Removed one node from the cluster. Queried the view with stale=false again while rebalance was in progress. View returned partial result:

      {"rows":[

      {"key":null,"value":88693}

      ],
      "errors":[
      {"from":"http://10.17.1.200:9503/_view_merge/?stale=false","reason":"{<<\"noproc\">>,<<\"{gen_server,call,[<0.2544.0>,

      {add,<0.6822.0>}

      ,infinity]}\">>}"}
      ]}

      This state is permanent: the view keeps returning partial result with this error. There are plenty of compaction_loop_died errors in the log on the failed node:

      =========================CRASH REPORT=========================
      crasher:
      initial call: couchbase_compaction_daemon:init/1
      pid: <0.12968.0>
      registered_name: couchbase_compaction_daemon
      exception exit: {compaction_loop_died,
      {noproc,
      {gen_server,call,
      [<0.2544.0>,

      {add,<0.12969.0>}

      ,infinity]}}}
      in function gen_server:terminate/6
      ancestors: [<0.12967.0>,ns_server_sup,ns_server_cluster_sup,<0.42.0>]
      messages: []
      links: [<0.12967.0>]
      dictionary: []
      trap_exit: true
      status: running
      heap_size: 233
      stack_size: 24
      reductions: 1074
      neighbours:

      Attaching logs and diag from the node where the error happens.

      1. logs.tar.bz2
        106 kB
        Aliaksey Artamonau
      2. ns-diag-20120124121218.txt.bz2
        514 kB
        Aliaksey Artamonau
      3. ns-diag-20120125183105.txt.bz2
        347 kB
        Aliaksey Artamonau
      4. reopened_diag2.bz2
        521 kB
        Aliaksey Artamonau
      5. reopened_log1.bz2
        188 kB
        Aliaksey Artamonau
      6. reopened_log2.bz2
        187 kB
        Aliaksey Artamonau
      7. verbose_log.tar.bz2
        2.84 MB
        Aliaksey Artamonau
      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        farshid Farshid Ghods (Inactive) added a comment -

        incremental rebalance out with reduce

        Show
        farshid Farshid Ghods (Inactive) added a comment - incremental rebalance out with reduce
        Hide
        Aliaksey Artamonau Aliaksey Artamonau added a comment -

        It's still reproducible. Aliaksey K is looking into it.

        Show
        Aliaksey Artamonau Aliaksey Artamonau added a comment - It's still reproducible. Aliaksey K is looking into it.
        Hide
        FilipeManana Filipe Manana (Inactive) added a comment -

        Try querying now with ?debug=true. It will help diagnose.

        Show
        FilipeManana Filipe Manana (Inactive) added a comment - Try querying now with ?debug=true. It will help diagnose.
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-couchdb-preview #339 (See http://qa.hq.northscale.net/job/github-couchdb-preview/339/)
        made notify_db_updated consume exits only from parent.MB-4686 (Revision df1e48bea718fc9d6f57feb00aa1a06a1d8e124c)

        Result = SUCCESS
        Aliaksey Kandratsenka :
        Files :

        • src/couchdb/couch_db_updater.erl
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-couchdb-preview #339 (See http://qa.hq.northscale.net/job/github-couchdb-preview/339/ ) made notify_db_updated consume exits only from parent. MB-4686 (Revision df1e48bea718fc9d6f57feb00aa1a06a1d8e124c) Result = SUCCESS Aliaksey Kandratsenka : Files : src/couchdb/couch_db_updater.erl
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        We believe it's fixed

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - We believe it's fixed

          People

          • Assignee:
            alkondratenko Aleksey Kondratenko (Inactive)
            Reporter:
            Aliaksey Artamonau Aliaksey Artamonau
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes