Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6468

Xdcr - Replication halts during rebalancing in 2 nodes at destination.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 2.0-beta
    • 2.0-beta
    • XDCR
    • Security Level: Public
    • None
    • 2.0-1648-rel
      1024 vbuckets
      Centos

    Description

      Setup a unidirectional replication between 2:2 node clusters from cluster1 to cluster2.
      Load 4M items on cluster1.
      Rebalance in 2 nodes at cluster2.

      • Start replication from cluster2 to cluster1
        -Rebalance is completed successfully on cluster2

      Error
      ---------

      • Replication has terminated on the clusters.

      Seeing this on the logs on the source side.

        • Reason for termination ==
        • {function_clause,[{couch_api_wrap,'get_missing_revs/2-fun-1',
          [400,
          [ {"Cache-Control","must-revalidate"},
          {"Content-Length","50"},
          {"Content-Type","application/json"},
          {"Date", "Tue, 28 Aug 2012 22:50:22 GMT"},
          {"Server", "MochiWeb/1.0 (Any of you quaids got a smint?)"}],
          {[{<<"error">>,<<"bad_request">>},
          {<<"reason">>, <<"not_my_vbucket">>}]}]},
          {xdc_vbucket_rep_worker,find_missing,2},
          {xdc_vbucket_rep_worker,queue_fetch_loop,4}]}

          [xdcr:error,2012-08-28T15:50:22.517,ns_1@10.3.121.32:<0.30832.38>:xdc_vbucket_rep:terminate:203]Shutting down without ever successfully initilizing: shutdown
          [xdcr:debug,2012-08-28T15:50:22.528,ns_1@10.3.121.32:<0.30597.38>:concurrency_throttle:signal_waiting:174]nothing to schedule, # of active reps: 168, # of acrtive target nodes: 1

          And


          error_logger:error,2012-08-28T16:03:32.555,ns_1@10.3.121.32:error_logger:ale_error_logger_handler:log_report:72]
          =========================CRASH REPORT=========================
          crasher:
          initial call: xdc_vbucket_rep:init/1
          pid: <0.25097.54>
          registered_name: []
          exception exit: {function_clause,
          [{couch_api_wrap,'get_missing_revs/2-fun-1',
          [400,
          [{"Cache-Control","must-revalidate"}

          ,

          {"Content-Length","50"}

          ,

          {"Content-Type","application/json"}

          ,

          {"Date","Tue, 28 Aug 2012 23:03:32 GMT"}

          ,

          {"Server", "MochiWeb/1.0 (Any of you quaids got a smint?)"}

          ],

          Unknown macro: {[{<<"error">>,<<"bad_request">>}, {<<"reason">>,<<"not_my_vbucket">>}]}

          ]},

          {xdc_vbucket_rep_worker,find_missing,2}

          ,

          {xdc_vbucket_rep_worker,queue_fetch_loop,4}

          ]}
          in function gen_server:terminate/6
          ancestors: [<0.25029.54>,<0.25025.54>,xdc_replication_sup,ns_server_sup,
          ns_server_cluster_sup,<0.60.0>]
          messages: [{'EXIT',<0.29078.54>,
          {function_clause,
          [{couch_api_wrap,'get_missing_revs/2-fun-1',

      Adding more logs.

      Attachments

        For Gerrit Dashboard: MB-6468
        # Subject Branch Project Status CR V

        Activity

          People

            junyi Junyi Xie (Inactive)
            ketaki Ketaki Gangal (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty