Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-46886

cluster_connect intermittently failing with 400 bad request error

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      GSI CI bot spawns a 4 node cluster and then tries to initialise two nodes using cluster_connect. At this phase, cluster_connect is frequently failing with below error:

       

      Traceback (most recent call last):
       File "./cluster_connect", line 108, in <module>
       main()
       File "./cluster_connect", line 103, in main
       cluster_run_lib.connect(**params)
       File "/opt/build/ns_server/pylib/cluster_run_lib.py", line 659, in connect
       addr, port), data).read()
       File "/usr/local/lib/python3.7/urllib/request.py", line 531, in open
       response = meth(req, response)
       File "/usr/local/lib/python3.7/urllib/request.py", line 641, in http_response
       'http', request, response, code, msg, hdrs)
       File "/usr/local/lib/python3.7/urllib/request.py", line 569, in error
       return self._call_chain(*args)
       File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain
       result = func(*args)
       File "/usr/local/lib/python3.7/urllib/request.py", line 649, in http_error_default
       raise HTTPError(req.full_url, code, msg, hdrs, fp)
      urllib.error.HTTPError: HTTP Error 400: Bad Request
      Connecting 2 nodes, bucket type membase, mem size 1500 with 1 replica copies, password asdasd, deployment plan {'n0': ['kv', 'n1ql'], 'n1': ['kv', 'index']}
      Connecting node 0 with services ['kv', 'n1ql']
      Connecting node 1 with services ['kv', 'index']
      Error 400: Bad Request (b'["Got HTTP status 500 from REST call post to https://127.0.0.1:19000/controller/addNodeV2. Body was: \\"[\\\\\\"Unexpected server error, request logged.\\\\\\"]\\""]')
      

      Attached complete set of logs (logs-functional_ns_server/logs) contains the logs of all nodes

      (Marking this as Critical as this issue is frequently failing the CI bot runs)

       

      Attachments

        1. fail.tar.gz
          738 kB
          Varun Velamuri
        2. logs-13.06.2021-23.25.tar.gz
          1.11 MB
          Varun Velamuri
        For Gerrit Dashboard: MB-46886
        # Subject Branch Project Status CR V

        Activity

          People

            artem Artem Stemkovski
            varun.velamuri Varun Velamuri
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty