Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-50950

[System Test][CBM] backup task failed with error - Received an unexpected error whilst streaming, beginning teardown: DCP stream closed unexpectedly, check the logs for more information

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 7.1.0
    • 7.1.0
    • sdkdev, tools
    • Untriaged
    • 1
    • No
    • Tools 2022-Feb

    Description

      7.1.0-2284

      Test:
      -test tests/integration/neo/test_neo_couchstore_milestone4.yml -scope tests/integration/neo/scope_couchstore.yml
      Scale 3 Iteration 3

      Backup task failed with error:

      2022-02-11T15:33:59.902-08:00 WARN: (DCP) (bucket4) (vb 87) Stream closed due to EOF being received while streaming, this could happen for a number of different reasons (e.g. non-graceful failover) | {"uuid":47872703404990,"snap_start":0,"snap_end":0,"last_seqno":0,"retries":0} -- couchbase.(*DCPAsyncWorker).End() at dcp_async_worker.go:538
      2022-02-11T15:33:59.903-08:00 WARN: (DCP) (bucket4) (vb 87) Received an unexpected error whilst streaming, beginning teardown: DCP stream closed unexpectedly, check the logs for more information -- couchbase.(*DCPAsyncWorker).handleDCPError() at dcp_async_worker.go:620
      

      During this time, test was doing a swap rebalance:

      [2022-02-11T15:30:26-08:00, sequoiatools/couchbase-cli:7.1:9cd967] server-add -c 172.23.108.103:8091 --server-add https://172.23.106.100 -u Administrator -p password --server-add-username Administrator --server-add-password password --services data
      [2022-02-11T15:30:44-08:00, sequoiatools/couchbase-cli:7.1:ab1d8f] rebalance -c 172.23.108.103:8091 --server-remove 172.23.104.5 -u Administrator -p password
      [2022-02-11T15:39:18-08:00, sequoiatools/cmd:abfebc] 60
      [2022-02-11T15:40:24-08:00, sequoiatools/cmd:9e6fc5] 600

      Cluster config:

      ########## Cluster config ##################
      ######  fts : 2 ===== > [172.23.105.111:8091 172.23.96.148:8091]  ###########
      ######  cbas : 3 ===== > [172.23.105.168:8091 172.23.106.188:8091 172.23.123.28:8091]  ###########
      ######  n1ql : 2 ===== > [172.23.104.137:8091 172.23.104.157:8091]  ###########
      ######  index : 6 ===== > [172.23.104.155:8091 172.23.104.70:8091 172.23.120.245:8091 172.23.121.117:8091 172.23.96.251:8091 172.23.96.252:8091]  ###########
      ######  eventing : 3 ===== > [172.23.104.67:8091 172.23.120.107:8091 172.23.98.135:8091]  ###########
      ######  backup : 1 ===== > [172.23.104.69:8091]  ###########
      ######  kv : 11 ===== > [172.23.105.107:8091 172.23.106.100:8091 172.23.108.103:8091 172.23.96.253:8091 172.23.97.119:8091 172.23.97.121:8091 172.23.97.122:8091 172.23.97.239:8091 172.23.99.20:8091 172.23.99.21:8091 172.23.99.25:8091]  ###########
      

      Attaching backup logs

      Server logs:
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.104.137.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.104.155.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.104.157.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.104.5.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.104.67.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.104.69.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.104.70.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.105.107.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.105.111.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.105.168.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.106.100.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.106.188.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.108.103.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.120.245.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.121.117.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.96.148.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.96.251.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.96.252.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.96.253.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.97.119.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.97.121.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.97.122.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.97.239.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.98.135.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.99.11.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.99.20.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.99.21.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644629679/collectinfo-2022-02-12T013441-ns_1%40172.23.99.25.zip

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            arunkumar Arunkumar Senthilnathan (Inactive)
            arunkumar Arunkumar Senthilnathan (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty