Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6420

XDCR: Replication fails to complete

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 2.0-beta
    • 2.0
    • XDCR
    • Security Level: Public
    • None
    • Build 1636
      vBuckets=1024

    Description

      On build 1636, upon running the automated test for load_with_ops via unidirectional replication with a 2 cluster (2 node each) set up, as:
      ./testrunner -i clusters_2_nodes_2.ini -t xdcr.uniXDCR.unidirectional.load_with_ops,items=5000,rdirection=unidirection,ctopology=chain,doc-ops=create

      [Source: 10.3.2.54, 10.3.121.126]
      [Destination: 10.3.2.55, 10.3.121.127]

      Replication doesn't complete on the destination cluster, causing verification to fail as well.
      Attached are the diags obtained from the 4 nodes.

      On build 1621 however, replication completes successfully.

      The Crash Report, on build 1636:

        • Reason for termination ==
        • {badarg,
          [
          Unknown macro: {erlang,monitor,[process,{xdc_rdoc_replication_srv,'ns_1@127.0.0.1'}]},
          {cb_generic_replication_srv,'-replicate_newnodes_docs/1-lc$^0/1-0-',2},
          {cb_generic_replication_srv,replicate_newnodes_docs,1},
          {cb_generic_replication_srv,handle_info,2},
          {gen_server,handle_msg,5},
          {proc_lib,init_p_do_apply,3}]}

          [ns_server:info,2012-08-24T18:33:14.781,ns_1@10.3.2.55:dist_manager:dist_manager:save_address_config:108]saving ip config to "/opt/couchbase/var/lib/couchbase/ip"
          [ns_server:debug,2012-08-24T18:33:14.782,ns_1@10.3.2.55:xdc_rdoc_replication_srv:cb_generic_replication_srv:handle_info:144]doing replicate_newnodes_docs
          [ns_server:debug,2012-08-24T18:33:14.784,ns_1@10.3.2.55:dist_manager:dist_manager:handle_call:225]save_address_config: ok
          [ns_server:debug,2012-08-24T18:33:14.784,ns_1@10.3.2.55:mb_mnesia:mb_mnesia:handle_call:133]Renaming node from 'ns_1@127.0.0.1' to 'ns_1@10.3.2.55'.
          [cluster:debug,2012-08-24T18:33:14.784,ns_1@10.3.2.55:ns_config:mb_mnesia:rename_node_in_config:158]renaming node conf {node,'ns_1@127.0.0.1',config_version} -> {node, 'ns_1@10.3.2.55', config_version}:
          {2,0} ->
          {2,0}
          [cluster:debug,2012-08-24T18:33:14.784,ns_1@10.3.2.55:ns_config:mb_mnesia:rename_node_in_config:158]renaming node conf nodes_wanted -> nodes_wanted:
          ['ns_1@127.0.0.1'] ->
          ['ns_1@10.3.2.55']
          [cluster:debug,2012-08-24T18:33:14.784,ns_1@10.3.2.55:ns_config:mb_mnesia:rename_node_in_config:158]renaming node conf {node,'ns_1@127.0.0.1',capi_port} -> {node, 'ns_1@10.3.2.55', capi_port}:
          8092 ->
          8092
          [error_logger:error,2012-08-24T18:33:14.784,ns_1@10.3.2.55:error_logger:ale_error_logger_handler:log_report:72]
          =========================CRASH REPORT=========================
          crasher:
          initial call: cb_generic_replication_srv:init/1
          pid: <0.451.0>
          registered_name: xdc_rdoc_replication_srv
          exception exit: {badarg,
          [{erlang,monitor,
          [process,
          {xdc_rdoc_replication_srv,'ns_1@127.0.0.1'}]}

          ,

          {cb_generic_replication_srv, '-replicate_newnodes_docs/1-lc$^0/1-0-',2}

          ,

          {cb_generic_replication_srv,replicate_newnodes_docs, 1}

          ,

          {cb_generic_replication_srv,handle_info,2}

          ,

          {gen_server,handle_msg,5}

          ,

          {proc_lib,init_p_do_apply,3}

          ]}
          in function gen_server:terminate/6
          ancestors: [ns_server_sup,ns_server_cluster_sup,<0.60.0>]
          messages: []
          links: [<0.332.0>,<0.452.0>]
          dictionary: []
          trap_exit: false
          status: running
          heap_size: 987
          stack_size: 24
          reductions: 1926
          neighbours:

      [cluster:debug,2012-08-24T18:33:14.784,ns_1@10.3.2.55:ns_config:mb_mnesia:rename_node_in_config:158]renaming node conf

      {node,'ns_1@127.0.0.1',compaction_daemon}

      ->

      {node, 'ns_1@10.3.2.55', compaction_daemon}

      :
      [

      {check_interval,30},{min_file_size,131072}] ->
      [{check_interval,30}

      ,

      {min_file_size,131072}

      ]
      [cluster:debug,2012-08-24T18:33:14.785,ns_1@10.3.2.55:ns_config:mb_mnesia:rename_node_in_config:158]renaming node conf

      {node,'ns_1@127.0.0.1',isasl}

      ->

      {node,'ns_1@10.3.2.55', isasl}

      :
      [

      {path,"/opt/couchbase/var/lib/couchbase/data/isasl.pw"}] ->
      [{path,"/opt/couchbase/var/lib/couchbase/data/isasl.pw"}

      ]
      [cluster:debug,2012-08-24T18:33:14.785,ns_1@10.3.2.55:ns_config:mb_mnesia:rename_node_in_config:158]renaming node conf

      {node,'ns_1@127.0.0.1',membership}

      ->

      {node, 'ns_1@10.3.2.55', membership}

      :
      active ->
      active
      [error_logger:error,2012-08-24T18:33:14.785,ns_1@10.3.2.55:error_logger:ale_error_logger_handler:log_report:72]

      Attachments

        For Gerrit Dashboard: MB-6420
        # Subject Branch Project Status CR V

        Activity

          People

            junyi Junyi Xie (Inactive)
            abhinav Abhi Dangeti
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty