Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-8294

Destination node with XDCR (heavy DGM) goes into pending.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Major
    • 2.1.0
    • 2.1.0
    • ns_server
    • Security Level: Public
    • None
    • Centos 64-bit

    Description

      Destination node with XDCR (heavy DGM) goes into pending.

      Setup Information:

      • Cluster Config : 4 nodes, OS: Centos 6.3, CPU : 6 Core, RAM : 16G, Disk : 500G
      • XDCR Topology : Unidirectional ( Master(3) -> Slave(1))
      • 4 buckets AbRegNums(2GB), MsgsCalls(2GB), RevAB(3GB), UserInfo(2GB)
      • Data Loaded using Viber Workload with 6.7M(RR ~45%), 0.1M(RR 100%), 11M(RR 100%), 12M(RR ~30%) data on the 4 buckets
      • Unidirectional XDCR setup to 1 node(master -> slave) for 3 buckets AbRegNums, RevAB, UserInfo

      Destination node goes into pending state(screenshot attached). Both beam.smp and memcached are running on the node.

      There are lot of crash reports like below but there are no views/design docs in the system.

      [error_logger:error,2013-05-16T1:40:30.265,ns_1@127.0.0.1:error_logger<0.6.0>:ale_error_logger_handler:log_report:72]
      =========================CRASH REPORT=========================
      crasher:
      initial call: set_view_update_daemon:init/1
      pid: <0.9511.707>
      registered_name: set_view_update_daemon
      exception exit: {noproc,
      {gen_server,call,
      ['capi_set_view_manager-UserInfo',

      {foreach_doc, #Fun<capi_ddoc_replication_srv.2.102018441>}

      ,
      infinity]}}
      in function gen_server:terminate/6
      ancestors: [ns_server_sup,ns_server_cluster_sup,<0.58.0>]
      messages: []
      links: [<0.298.0>,<0.9512.707>]
      dictionary: []
      trap_exit: false
      status: running
      heap_size: 121393
      stack_size: 24
      reductions: 7537
      neighbours:

      =========================CRASH REPORT=========================
      crasher:
      initial call: compaction_daemon:spawn_bucket_compactor/3-fun-2/0
      pid: <0.9508.707>
      registered_name: []
      exception exit: {noproc,
      {gen_server,call,
      ['capi_set_view_manager-RevAB',

      {foreach_doc, #Fun<capi_ddoc_replication_srv.1.36030090>}

      ,
      infinity]}}
      in function gen_server:call/3
      in call from capi_ddoc_replication_srv:foreach_live_ddoc_id/2
      in call from capi_ddoc_replication_srv:fetch_ddoc_ids/1
      in call from compaction_daemon:'spawn_bucket_compactor/3-fun-2'/4
      ancestors: [compaction_daemon,ns_server_sup,ns_server_cluster_sup,
      <0.58.0>]
      messages: []
      links: [<0.412.0>]
      dictionary: []
      trap_exit: false
      status: running
      heap_size: 28657
      stack_size: 24
      reductions: 30607
      neighbours:

      The pending node is in the same state and can be used for investigation:
      coconut-h20804.hq.couchbase.com

      Attachments

        1. Buckets.png
          Buckets.png
          142 kB
        2. Server.png
          Server.png
          135 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            deepkaran.salooja Deepkaran Salooja
            deepkaran.salooja Deepkaran Salooja
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty