Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-18534

Fix Array Indexing Query Performance

    XMLWordPrintable

Details

    • Bug
    • Resolution: Incomplete
    • Critical
    • 4.5.0
    • 4.5.0
    • query
    • Triaged
    • No

    Description

      Recent Array Indexing results have been published here:
      http://showfast.sc.couchbase.com/#/timeline

      Please review Q2, stale=ok results.
      If you see title has compaction_mode = full, this means circular compaction is disabled.
      For your reference, this is the html report:
      http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=hades_450-1613-enterprise_a66_access

      A few things I noticed:
      We noticed that avg query latency in Array Q2(stale=ok) is close to 100ms.
      For comparison, for Q2(stale=ok), avg query latency is 8~10ms.

      A few things we noticed in the showfast. (Only focused on Q2, stale=ok results now)
      Indexer CPU utilization is relatively low. For array indexing Q2(stale=ok), it is 2 cores. For Q2(stale=ok), it is close to 12 cores.
      Avg scan latency is 20ms in the query log. But in show fast, avg query latency is 100ms.

      John's analysis here;
      ===
      Look at indexer CPU graph, it does not look like there is a lot GC going on. So it is not clear how the next 2i build will help (with GC optimization).
      >> Indexer CPU utilization is relatively low. For array indexing Q2(stale=ok), it is 2 cores. For Q2(stale=ok), it is close to 12 cores.
      Since throughput is more than 10x slower than Q2, so I would expect indexer CPU utilization is lower as well.

      "Keshav, cbq cpu could be maxed out at 24 cores (note that you have hyper threading turned off in this cluster). "
      ====

      Question for the query team:
      Based on the query log, the avg index scan latency is only 20ms, why avg query latency is 100ms?

      This is the index/query definition:
      https://github.com/couchbase/perfrunner/blob/master/tests/n1ql/n1ql_thr_array_indexing_Q2_20M_gsi_ok.test.

      This is one sample run:
      http://perf.jenkins.couchbase.com/job/hades/2915/

      Please let me know if you need anything else.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            keshav Keshav Murthy
            qi Qi Zhu (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty