Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-43986

[Sanity] intermittent index scan timeout

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      As part of sanity we have a simple ADVISOR test doing a query but about 25% of the run will fail as follow:

       [2021-01-29 13:21:21,825] - [tuq:1415] INFO - RUN QUERY SELECT airportname FROM `travel-sample` WHERE type = "airport" AND lower(city) = "lyon" AND country = "France"
      [2021-01-29 13:21:21,825] - [rest_client:3872] INFO - query params : statement=SELECT+airportname+FROM+%60travel-sample%60+WHERE+type+%3D+%22airport%22+AND+lower%28city%29+%3D+%22lyon%22+AND+country+%3D+%22France%22&scan_consistency=REQUEST_PLUS
      [2021-01-29 13:25:21,929] - [rest_client:1016] ERROR - POST http://172.23.120.100:8093/query?statement=SELECT+airportname+FROM+%60travel-sample%60+WHERE+type+%3D+%22airport%22+AND+lower%28city%29+%3D+%22lyon%22+AND+country+%3D+%22France%22&scan_consistency=REQUEST_PLUS body:  headers: {'Authorization': 'Basic am9hb0RvZTpwYXNzd29yZDE='} error: 500 reason: unknown b'{\n"requestID": "0513bff8-876d-4ec0-ba78-31905e1bc254",\n"signature": {"airportname":"json"},\n"results": [\n],\n"errors": [{"code":5000,"msg":" Index scan timed out from [127.0.0.1:9101] - cause:  Index scan timed out from [127.0.0.1:9101]"}],\n"status": "errors",\n"metrics": {"elapsedTime": "4m0.100544433s","executionTime": "4m0.100475114s","resultCount": 0,"resultSize": 0,"serviceLoad": 1,"errorCount": 1}\n}\n' auth: joaoDoe:password1
      

      This issue mainly happens on DEBIAN on regular basis but it has also be seen (less often) on SUSE: http://server.jenkins.couchbase.com/job/build_sanity_matrix/DISTRO=suse12,TYPE=1node/8618/.

      Links to DEBIAN failing job: http://server.jenkins.couchbase.com/job/build_sanity_matrix/DISTRO=debian9,TYPE=1node/8634/

      Link to DEBIAN logs: http://server.jenkins.couchbase.com/job/build_sanity_matrix/DISTRO=debian9,TYPE=1node/8634/artifact/logs/testrunner-21-Jan-29_13-09-56/test_18/172.23.120.100-20210129-1325-diag.zip

      This is intermittent for example history of failing and passing jobs:

       

       

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            amit.kulkarni Amit Kulkarni
            pierre.regazzoni Pierre Regazzoni
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty