Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-44881

[System Test] HTTP status 500 received from engageCluster2 API multiple times

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 7.0.0
    • Cheshire-Cat
    • ns_server
    • Untriaged
    • 1
    • Yes

    Description

      7.0.0-4655

      Test:
      -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas.yml -scope tests/integration/cheshirecat/scope_cheshirecat_with_backup.yml
      Scale 2
      Iteration 1

      Following errors observed in console:

      [2021-03-10T17:00:40-08:00, sequoiatools/couchbase-cli:7.0:afafc5] server-add -c 172.23.97.74:8091 --server-add https://172.23.96.14 -u Administrator -p password --server-add-username Administrator --server-add-password password --services eventing
       
      Error occurred on container - sequoiatools/couchbase-cli:7.0:[server-add -c 172.23.97.74:8091 --server-add https://172.23.96.14 -u Administrator -p password --server-add-username Administrator --server-add-password password --services eventing]
       
      docker logs afafc5
      docker start afafc5
       
      �ERROR: Prepare join failed. Got HTTP status 500 from REST call post to https://172.23.96.14:18091/engageCluster2. Body was: "[\"Unexpected server error, request logged.\"]"
      

      From .97.74 debug log:

      {net_kernel,{'EXIT',<0.4272.182>,shutdown}}
      [cluster:debug,2021-03-10T17:00:41.246-08:00,ns_1@172.23.97.74:ns_cluster<0.236.0>:ns_cluster:do_add_node_with_connectivity:836]Reply from engage_cluster on {"172.23.96.14",18091}:
      {error,rest_error,
             <<"Got HTTP status 500 from REST call post to https://172.23.96.14:18091/engageCluster2. Body was: \"[\\\"Unexpected server error, request logged.\\\"]\"">>,
             {bad_status,500,<<"[\"Unexpected server error, request logged.\"]">>}}
      [cluster:debug,2021-03-10T17:00:41.246-08:00,ns_1@172.23.97.74:ns_cluster<0.236.0>:ns_cluster:handle_call:361]add_node(https, "172.23.96.14", 18091, <<"0">>, ..) -> {error,engage_cluster,
                                                              <<"Prepare join failed. Got HTTP status 500 from REST call post to https://172.23.96.14:18091/engageCluster2. Body was: \"[\\\"Unexpected server error, request logged.\\\"]\"">>,
                                                              {error,rest_error,
                                                               <<"Got HTTP status 500 from REST call post to https://172.23.96.14:18091/engageCluster2. Body was: \"[\\\"Unexpected server error, request logged.\\\"]\"">>,
                                                               {bad_status,500,
                                                                <<"[\"Unexpected server error, request logged.\"]">>}}}
      [user:info,2021-03-10T17:00:41.247-08:00,ns_1@172.23.97.74:<0.3911.182>:ns_cluster:add_node_to_group:89]Failed to add node 172.23.96.14:18091 to cluster. Prepare join failed. Got HTTP status 500 from REST call post to https://172.23.96.14:18091/engageCluster2. Body was: "[\"Unexpected server error, request logged.\"]"
      

      Logs:

      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.106.134.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.120.58.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.120.73.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.120.74.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.120.75.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.120.77.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.120.81.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.120.86.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.121.77.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.123.24.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.123.25.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.123.26.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.123.31.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.123.32.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.123.33.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.96.122.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.96.243.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.96.254.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.105.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.110.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.112.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.148.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.149.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.150.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.151.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.241.zip
      https://cb-jira.s3.us-east-2.amazonaws.com/logs/addnode_err500/collectinfo-2021-03-11T011802-ns_1%40172.23.97.74.zip

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              timofey.barmin Timofey Barmin
              arunkumar Arunkumar Senthilnathan (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty