Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-61534

XCDR tests hit "received nil response for unique-key" on build 7.6.2-3550

    XMLWordPrintable

Details

    • Untriaged
    • 0
    • Yes

    Description

      Several XDCR runs on build 7.6.2-3550 don't work correctly. We see timeout messages in the job logs.

      2024-04-16T04:37:23 [WARNING] XDCR sampling timed out after 600 seconds

      2024-04-16T04:47:45 [WARNING] XDCR sampling timed out after 600 seconds

      2024-04-16T04:58:06 [WARNING] XDCR sampling timed out after 600 seconds

      2024-04-16T05:08:28 [WARNING] XDCR sampling timed out after 600 seconds

      2024-04-16T05:18:50 [WARNING] XDCR sampling timed out after 600 seconds

      In ns_server.goxdcr.log, there are many error messages.

      2024-04-16T05:20:22.110-07:00 ERRO GOXDCR.XmemNozzle: pipelineFullTopic=00231cb2d2f898a399a8d45034e39a33/bucket-1/bucket-1, Received nil response for unique-key cd5a6d-00009269631610105%!s(MISSING)%!v(MISSING)

      Build: 7.6.2-3550

      Job: https://perf.jenkins.couchbase.com/job/titan_5node2/3422/

      Logs:

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-titan_5node2-3422/172.23.96.105.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-titan_5node2-3422/172.23.96.106.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-titan_5node2-3422/172.23.96.107.zip

       

      Job: https://perf.jenkins.couchbase.com/job/titan_5node1/3091/

      Logs: 

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-titan_5node1-3091/172.23.96.100.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-titan_5node1-3091/172.23.96.102.zip

      2024-04-15T20:10:22.545-07:00 ERRO GOXDCR.XmemNozzle: pipelineFullTopic=1b7d9f2b5337e632951c3b5c7ad51c52/bucket-1/bucket-1, xmem_1b7d9f2b5337e632951c3b5c7ad51c52/bucket-1/bucket-1_172.23.96.102:11210_2 batchGet received fatal error and had to abort. Expected 493 responses, got 7 responses. err=Connection is bad

      2024-04-15T20:10:22.545-07:00 INFO GOXDCR.XmemNozzle: pipelineFullTopic=1b7d9f2b5337e632951c3b5c7ad51c52/bucket-1/bucket-1, xmem_1b7d9f2b5337e632951c3b5c7ad51c52/bucket-1/bucket-1_172.23.96.102:11210_2 Expected=map[3416522752:[<ud>d2eb68-000000051204</ud> 1427 745 2024-04-15 20:10:22.498040419 -0700 PDT m=+1640.738805939 2] 3416522753:[<ud>d2eb68-000000059713</ud> 1445 745

      2024-04-15 20:10:22.4980492 -0700 PDT m=+1640.738814720 2] 3416522754:[<ud>d2eb68-000000027459</ud> 1511 745 2024-04-15 20:10:22.498051159 -0700 PDT m=+1640.738816681 2] 3416522755:[<ud>...truncated due to encoding error

      2024-04-15T20:10:22.546-07:00 ERRO GOXDCR.XmemNozzle: pipelineFullTopic=1b7d9f2b5337e632951c3b5c7ad51c52/bucket-1/bucket-1, Received nil response for unique-key d2eb68-0000000804923971%!s(MISSING)%!v(MISSING)

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            bo-chun.wang Bo-Chun Wang
            bo-chun.wang Bo-Chun Wang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty