Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11563

[System Test] Indexing stuck, seeing multiple crashes on couch_set_view_updater.erl with badmatch error

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Test Blocker
    • 3.0
    • 3.0
    • view-engine
    • Security Level: Public
    • None
    • Centos 6.4
      Live Cluster : 10.6.2.163:8091
      Build : 3.0.0-880-rel
    • Untriaged
    • Hide
      - memcached_log for vb-109
      - couchdb_badmatch-sampleoutput
      - cbcollectinfo from the nodes https://s3.amazonaws.com/bugdb/new+folder/11563.tar
      Show
      - memcached_log for vb-109 - couchdb_badmatch-sampleoutput - cbcollectinfo from the nodes https://s3.amazonaws.com/bugdb/new+folder/11563.tar
    • Unknown

    Description

      Setup:

      • 7 Nodes, 2 buckets, 1 ddoc,2 views
      • 20 M items ( no DGM)
      • rebalance in 1 node, with view-query 100 views/sec and 4-5k mix of upd-delete ops/sec across the cluster.
      • Stop load, wait for indexing to complete

      — Indexing keeps looping, making no progress beyond 2-3%.
      — The couchdb log shows multiple crashes on couch_set_view/src/couch_set_view_updater.erl with a bad-match error.
      "
      error:

      {badmatch,undefined}

      stacktrace: [{couch_upr_client,enum_docs_since,8,
      [

      {file, "/buildbot/build_slave/centos-5-x64-300-builder/build/build/couchdb/src/couch_upr/src/couch_upr_client.erl"}

      ,

      {line,166}

      ]},
      {couch_set_view_updater,'load_changes/8-fun-2',12,
      [

      {file, "/buildbot/build_slave/centos-5-x64-300-builder/build/build/couchdb/src/couch_set_view/src/couch_set_view_updater.erl"},
      {line,475}]},
      {lists,foldl,3,[{file,"lists.erl"},{line,1248}]},
      {couch_set_view_updater,load_changes,8,
      [{file, "/buildbot/build_slave/centos-5-x64-300-builder/build/build/couchdb/src/couch_set_view/src/couch_set_view_updater.erl"}

      ,

      {line,539}

      ]},
      {couch_set_view_updater,'update/8-fun-2',14,
      [

      {file, "/buildbot/build_slave/centos-5-x64-300-builder/build/build/couchdb/src/couch_set_view/src/couch_set_view_updater.erl"}

      ,

      {line,266}

      ]}]

      [couchdb:error,2014-06-26T14:51:12.185,ns_1@10.6.2.163:<0.28486.6>:couch_log:error:42]Set view `saslbucket`, main (prod) group `_design/ddoc2`, received error from updater: {badmatch,
      "

      • The memcached log shows for the same vb-109(say) , it keep requesting for same seq nos
        Thu Jun 26 14:40:26.327523 PDT 3: (saslbucket) UPR (Producer) eq_uprq:mapreduce_view: saslbucket _design/ddoc2 (prod/main) - (vb 109) stream created with start seqno 30673 and end seqno 30721

      Attachments

        Activity

          People

            ketaki Ketaki Gangal (Inactive)
            ketaki Ketaki Gangal (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty