Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-43070

[System Test] : query service (cbq-engine) not responding

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Test Blocker
    • 7.0.0
    • Cheshire-Cat
    • query

    Description

      Build : 7.0.0-3875
      Test : -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll.yml -scope tests/integration/cheshirecat/scope_cheshirecat.yml
      Scale : 3
      Iteration : 1 (day 1)

      In the longevity test with collection support, there are steps to run N1QL queries continuously against collections. Concurrently, queries are also run on buckets. And there are also threads that run in parallel that will create indexes on collections of other buckets.

      Both the query nodes get hung after a while and stop responding. Tried accessing QWB and cbq-engine - it does not respond. Same result issuing a query via REST.

      Attached are the CPU/Heap/Memory profiles and active/completed requests and goroutine dumps from both the query nodes - 172.23.96.251 & 172.23.96.252. Also have specified the location of the cbcollect logs.

      Cluster Config :
      Eventing : 3 ===== > [172.23.104.137:8091 172.23.121.3:8091 172.23.123.27:8091]
      FTS : 2 ===== > [172.23.104.157:8091 172.23.120.245:8091]
      KV : 11 ===== > [172.23.104.5:8091 172.23.108.103:8091 172.23.96.148:8091 172.23.97.119:8091 172.23.97.121:8091 172.23.97.122:8091 172.23.97.239:8091 172.23.98.135:8091 172.23.99.20:8091 172.23.99.21:8091 172.23.99.25:8091]
      Index : 6 ===== > [172.23.104.67:8091 172.23.104.69:8091 172.23.104.70:8091 172.23.123.28:8091 172.23.96.253:8091 172.23.96.95:8091]
      CBAS : 3 ===== > [172.23.106.188:8091 172.23.97.242:8091 172.23.99.11:8091]
      N1QL : 2 ===== > [172.23.96.251:8091 172.23.96.252:8091]

      Attachments

        1. goroutine_dump_251.txt
          26.87 MB
        2. goroutine_dump_252.txt
          28.83 MB
        3. heap_251.prof
          182 kB
        4. heap_252.prof
          193 kB
        5. query_251.mprof
          23.40 MB
        6. query_252.mprof
          24.14 MB
        7. query_active_requests_251.json
          23 kB
        8. query_active_requests_252.json
          18 kB
        9. query_completed_requests_251.json
          182 kB
        10. query_completed_requests_252.json
          208 kB
        11. query_cpu_profile_251.pprof
          43 kB
        12. query_cpu_profile_252.pprof
          42 kB
        13. query_dumps0109.zip
          10.30 MB

        Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. MB-43486 Leak due to verbose logging MultiScanInternal()

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              marco.greco Marco Greco (Inactive)
              mihir.kamdar Mihir Kamdar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty