Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-14813

GoXDCR: Frequent "Failed to resend document" errors result in very slow replication

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 4.0.0
    • 4.0.0
    • XDCR
    • Security Level: Public

    Description

      Build


      4.0.0-2036

      -found during manual testing

      -simple uni-xdcr with checkpoint_interval =60s
      -loaded 37840 keys on src. All keys got replicated
      -flushed target bucket
      -after a min, all keys got replicated
      -flushed target bucket again
      -replication very slow and is seen in bursts (37k keys take more than 5-6 mins), see pipeline getting constructed with errors like

      2015-05-04 17:25:00 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.189:11210_1:Failed to resend document aruna3.9482, has tried to resend it 6, maximum retry 5 reached]
      2015-05-04 17:23:41 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.189:11210_1:Failed to resend document aruna3.18083, has tried to resend it 6, maximum retry 5 reached]
      2015-05-04 17:22:22 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.188:11210_1:Failed to resend document aruna3.6843, has tried to resend it 6, maximum retry 5 reached]
      2015-05-04 17:21:03 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.189:11210_1:Failed to resend document aruna3.7037, has tried to resend it 6, maximum retry 5 reached]
      2015-05-04 17:19:43 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.188:11210_1:Failed to resend document aruna3.6191, has tried to resend it 6, maximum retry 5 reached]
      2015-05-04 17:18:24 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.188:11210_1:Failed to resend document aruna3.33717, has tried to resend it 6, maximum retry 5 reached]
      2015-05-04 17:17:04 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.189:11210_1:Failed to resend document aruna3.12977, has tried to resend it 6, maximum retry 5 reached]
      2015-05-04 17:16:49 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.189:11210_1:Failed to resend document aruna3.25659, has tried to resend it 6, maximum retry 5 reached]
      2015-05-04 17:15:30 map[xmem_672645dcb5e47d01f46a75ac0c187d2c/default/default_10.3.4.189:11210_0:Failed to resend document aruna3.37792, has tried to resend it 6, maximum retry 5 reached]

      Why do we see this error? I see this in many scenarios. There was no load or topology change on target cluster except for the flush.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              apiravi Aruna Piravi (Inactive)
              apiravi Aruna Piravi (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty