Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7525

Add view engine utilization stats per design document and per node (useful for rebalance performance tests for example)

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0, 2.0.1
    • Fix Version/s: 2.0.1
    • Component/s: view-engine
    • Security Level: Public
    • Labels:
      None

      Description

      Using a special URL, it's possible to get view engine utilization
      stats per design document (index), and per node.
      These stats consist on the total indexing time (sum of indexing times
      for all index updater runs), total compaction time, number of index
      updates, number of compactions, etc.

      At any time, it's possible to ask to reset these stats.
      This is useful for rebalance performance/analysus tests, for example,
      right after an index is created and before the rebalance it triggered,
      a test can ask to reset the stats and after the rebalance finishes
      it queries the stats - these allows the test to know how much of
      the total rebalance time was spent on indexing and index compaction
      per design document and per node.

      Note that these stats are kept in memory, not persisted to disk.
      This means that after a node is restarted, all the stats values
      are zeroed.

      Examples:

      1. Get current utilization stats for design document A, bucket 'default'

      $ curl -s http://192.168.1.2:9500/_set_view/default/_design/A/_get_utilization_stats | json_xs

      { "updater_interruptions" : 0, "compactor_interruptions" : 0, "useful_indexing_time" : 82.364292, "updates" : 1, "wasted_indexing_time" : 0, "total_indexing_time" : 82.364292, "compactions" : 0, "compaction_time" : 0 }
      1. Reset the utilization stats for the previous design document and bucket.
      2. Success return HTTP code is 201

      $ curl -X POST http://192.168.1.2:9500/_set_view/default/_design/A/_reset_utilization_stats
      true

      1. Get the utilization stats again.

      $ curl -s http://192.168.1.2:9500/_set_view/default/_design/A/_get_utilization_stats | json_xs

      { "updater_interruptions" : 0, "compactor_interruptions" : 0, "useful_indexing_time" : 0, "updates" : 0, "wasted_indexing_time" : 0, "total_indexing_time" : 0, "compactions" : 0, "compaction_time" : 0 }

      These allows stigmatization of the analysis done at:
      https://docs.google.com/document/d/1eUV53B5pXj-5X5FPcbOA3jz1U6jOjh1hnZmZa9bUNs0/edit

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Show
        FilipeManana Filipe Manana (Inactive) added a comment - http://review.couchbase.org/#/c/23900/
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-couchdb-preview #551 (See http://qa.hq.northscale.net/job/github-couchdb-preview/551/)
        MB-7525 Add utilization stats for set views (per ddoc) (Revision 59896075da342940a5b6a75147bbccca0148037a)

        Result = SUCCESS
        Filipe David Borba Manana :
        Files :

        • src/couch_set_view/src/couch_set_view.erl
        • src/couch_set_view/src/couch_set_view_group.erl
        • src/couch_set_view/test/25-util-stats.t
        • src/couch_set_view/Makefile.am
        • src/couch_set_view/src/couch_set_view_http.erl
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-couchdb-preview #551 (See http://qa.hq.northscale.net/job/github-couchdb-preview/551/ ) MB-7525 Add utilization stats for set views (per ddoc) (Revision 59896075da342940a5b6a75147bbccca0148037a) Result = SUCCESS Filipe David Borba Manana : Files : src/couch_set_view/src/couch_set_view.erl src/couch_set_view/src/couch_set_view_group.erl src/couch_set_view/test/25-util-stats.t src/couch_set_view/Makefile.am src/couch_set_view/src/couch_set_view_http.erl
        Hide
        FilipeManana Filipe Manana (Inactive) added a comment -

        Merged

        Show
        FilipeManana Filipe Manana (Inactive) added a comment - Merged
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-couchdb-preview #552 (See http://qa.hq.northscale.net/job/github-couchdb-preview/552/)
        MB-7525 Reset replica index utilization stats as well (Revision 7e2e260003fc1b3eef0fff9ce938fecbc648f915)

        Result = SUCCESS
        Filipe David Borba Manana :
        Files :

        • src/couch_set_view/src/couch_set_view_group.erl
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-couchdb-preview #552 (See http://qa.hq.northscale.net/job/github-couchdb-preview/552/ ) MB-7525 Reset replica index utilization stats as well (Revision 7e2e260003fc1b3eef0fff9ce938fecbc648f915) Result = SUCCESS Filipe David Borba Manana : Files : src/couch_set_view/src/couch_set_view_group.erl

          People

          • Assignee:
            FilipeManana Filipe Manana (Inactive)
            Reporter:
            FilipeManana Filipe Manana (Inactive)
          • Votes:
            1 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes