Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-43070

[System Test] : query service (cbq-engine) not responding

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Test Blocker
    • 7.0.0
    • Cheshire-Cat
    • query

    Description

      Build : 7.0.0-3875
      Test : -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll.yml -scope tests/integration/cheshirecat/scope_cheshirecat.yml
      Scale : 3
      Iteration : 1 (day 1)

      In the longevity test with collection support, there are steps to run N1QL queries continuously against collections. Concurrently, queries are also run on buckets. And there are also threads that run in parallel that will create indexes on collections of other buckets.

      Both the query nodes get hung after a while and stop responding. Tried accessing QWB and cbq-engine - it does not respond. Same result issuing a query via REST.

      Attached are the CPU/Heap/Memory profiles and active/completed requests and goroutine dumps from both the query nodes - 172.23.96.251 & 172.23.96.252. Also have specified the location of the cbcollect logs.

      Cluster Config :
      Eventing : 3 ===== > [172.23.104.137:8091 172.23.121.3:8091 172.23.123.27:8091]
      FTS : 2 ===== > [172.23.104.157:8091 172.23.120.245:8091]
      KV : 11 ===== > [172.23.104.5:8091 172.23.108.103:8091 172.23.96.148:8091 172.23.97.119:8091 172.23.97.121:8091 172.23.97.122:8091 172.23.97.239:8091 172.23.98.135:8091 172.23.99.20:8091 172.23.99.21:8091 172.23.99.25:8091]
      Index : 6 ===== > [172.23.104.67:8091 172.23.104.69:8091 172.23.104.70:8091 172.23.123.28:8091 172.23.96.253:8091 172.23.96.95:8091]
      CBAS : 3 ===== > [172.23.106.188:8091 172.23.97.242:8091 172.23.99.11:8091]
      N1QL : 2 ===== > [172.23.96.251:8091 172.23.96.252:8091]

      Attachments

        1. goroutine_dump_251.txt
          26.87 MB
          Mihir Kamdar
        2. goroutine_dump_252.txt
          28.83 MB
          Mihir Kamdar
        3. heap_251.prof
          182 kB
          Mihir Kamdar
        4. heap_252.prof
          193 kB
          Mihir Kamdar
        5. query_251.mprof
          23.40 MB
          Mihir Kamdar
        6. query_252.mprof
          24.14 MB
          Mihir Kamdar
        7. query_active_requests_251.json
          23 kB
          Mihir Kamdar
        8. query_active_requests_252.json
          18 kB
          Mihir Kamdar
        9. query_completed_requests_251.json
          182 kB
          Mihir Kamdar
        10. query_completed_requests_252.json
          208 kB
          Mihir Kamdar
        11. query_cpu_profile_251.pprof
          43 kB
          Mihir Kamdar
        12. query_cpu_profile_252.pprof
          42 kB
          Mihir Kamdar

        Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. MB-43486 Leak due to verbose logging MultiScanInternal()

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              marco.greco Marco Greco (Inactive)
              mihir.kamdar Mihir Kamdar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty