Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-13501

GoXDCR: Source does not pre-replicate after crash

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Critical
    • 4.0.0
    • 4.0.0
    • XDCR
    • Security Level: Public
    • Untriaged
    • No

    Description

      Build
      -------
      3.5.0-1270

      • Seeing multiple Checkpointing failed errors after crash.
      • No new pre-replicate posts recorded on .188(target)
      • In erlang-xdcr, the source would pre-replicate to target cluster as soon as it recovers from a crash, regardless of whether it has received mutations.

      Testcase
      ./testrunner -i xdcr.ini enable_goxdcr=True,checkpoint_interval=60 -t xdcr.checkpointXDCR.XDCRCheckpointUnitTest.test_source_node_crash,rdirection=unidirection,topology=chain,replication_type=xmem

      StatisticsManager14:02:03.624229 [INFO] Dcp dcp_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.186:11210_0 streamed 0 items. [] streams inactive
      StatisticsManager14:02:03.624292 [INFO] Rounter Router_dcp_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.186:11210_1 = map[xmem_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.188:11210_1:0]
      StatisticsManager14:02:03.624360 [INFO] Dcp dcp_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.186:11210_1 streamed 0 items. [] streams inactive
      ToplogyChangeDetector14:02:03.711611 [INFO] validateSourceTopology...
      ToplogyChangeDetector14:02:13.705008 [INFO] validateSourceTopology...
      XmemNozzle14:02:19.907819 [INFO] Send batch count=1
      ToplogyChangeDetector14:02:23.717340 [INFO] validateSourceTopology...
      ToplogyChangeDetector14:02:33.692710 [INFO] validateSourceTopology...
      ToplogyChangeDetector14:02:43.725049 [INFO] validateSourceTopology...
      ToplogyChangeDetector14:02:53.675109 [INFO] validateSourceTopology...
      StatisticsManager14:03:03.570084 [INFO] Stats for pipeline replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default-654955431 {"CkptMgr": {"num_checkpoints": 1, "num_failedckpts": 0, "time_committing": {"count": 1, "max": 0, "mean": 0, "min": 0}}, "Errors": ["map[CheckpointMgr:Checkpointing failed]","map[xmem_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.188:11210_1:MCResponse status=0x08, opcode=0xa0, opaque=0, msg: ]","","","","","","","",""], "Overview":

      {"bandwidth_usage": 1.0833333333333333, "changes_left": 0, "data_replicated": 65, "docs_checked": 0, "docs_filtered": 0, "docs_opt_repd": 1, "docs_received_from_dcp": 1, "docs_written": 1, "num_checkpoints": 1, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0.016666666666666666, "rate_replicated": 0.016666666666666666, "time_committing": 0, "wtavg_docs_latency": 789}

      , "Status": "Replicating", "XDCRRouter":

      {"docs_filtered": 0}

      , "dcp_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.186:11210_0":

      {"docs_received_from_dcp": 1}

      , "dcp_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.186:11210_1":

      {"docs_received_from_dcp": 0}

      , "xmem_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.188:11210_0": {"data_replicated": 65, "docs_opt_repd": 1, "docs_rep_queue":

      {"count": 1, "max": 1, "mean": 1, "min": 1}

      , "docs_written": 1, "size_rep_queue": {"count": 1, "max": 65, "mean": 65, "min": 65}}, "xmem_replicationSpec/f1ee1e55075c5a2922b85df16deeca96/default/default_10.3.4.188:11210_1": {"data_replicated": 0, "docs_opt_repd": 0, "docs_rep_queue":

      {"count": 0, "max": 0, "mean": 0, "min": 0}

      , "docs_written": 0, "size_rep_queue":

      {"count": 0, "max": 0, "mean": 0, "min": 0}

      }}

      Attaching logs.
      .186 -> .188

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            xiaomei Xiaomei Zhang (Inactive)
            apiravi Aruna Piravi (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty